Baking unsigned 8bit vertex lighting in Blender 4 

Now this is some real niche material. I have been experimenting a little with Dreamcast era OpenGL again, and wanting to incorporate some lighting effects. Here's a little unlit area from my prototype game 'Dashy No Blast'.

Here's the textured mesh without any lighting, shown in Blender.

A videogame screenshot. An opaque window on a stone brick wall.

By adding some lights to the scene and switching to 'Rendered' view, we get something like this:

A videogame screenshot. An opaque window on a stone brick wall. The scene is illuminated with light and shadows.

The goal is to now approximate this light and shadow by 'baking' the light to the vertex colour of the scene. We need to add vertex colour attributes to each mesh that we want to light. 

An inspector panel from Blender

Next, in the Render inspector tab, we configure Cycles to perform a Combined bake, using lighting and diffuse, with the output set to 'Active Color Attribute'.

Another Blender inspector panel showing Cycles render configuration

Then, by clicking 'Bake', the lighting will be calculated and baked to the vertex colours. To see this, we can configure viewport shading to colour the scene with 'Attribute', which shows the baked vertex lighting. Looks okay, but there is a hidden problem! 

Blender, showing the configuration of a viewmode to allow vertex colour to be displayed

When creating the colour attributes, I used the 'Color' data type. For some reason, this type allows colour values outside of the normalised range 0.0 to 1.0. Note here in the Blender spreadsheet viewer that the selected vertex has a red value of 1.082.

Blender spreadsheet view, inspecting a single vertex

This causes problems later - my pipeline converts these values to 8 bit unsigned 'byte' values, in the range 0 to 255, where an input value of 1.0 should result in an output of 255. An input of 1.082 results in an output of 20, because 1.082 * 255 = 275 point something, but an unsigned byte only holds a maximum value of 255 before it overflows back to zero. In practice this resulted in my illuminated orange window losing most of its red and looking like this:

The window is displayed using vertex lighting that has gone out of range, making it green

So we recreate or convert the colour attribute data, using the 'Byte Color' instead of 'Color' data type.

Blender 'convert color attribute' popup

Reinspecting one of my problem green vertices in the spreadsheet view, we now see that the red value is back in the valid range as 1.0.

Another blender spreadsheet view, showing valid values for a vertex

This results in a more orange window when displayed ingame, as desired.

The window now displays correctly

One final note: the ambient light level can be adjusted from the World inspector's 'Surface' dropdown. Remember to bake the light after changing this to see any difference.

Blender World inspector showing  the Surface Color and strength for ambient lighting


This article was updated on