Re-compiling shader in openGL

Question

I'm writing my own OpenGL-3D-Application and have stumbled across a little problem:

I want the number of light sources to be dynamic. For this, my shader contains an array of my lights struct:uniform PointLight pointLights[NR_POINT_LIGHTS];

The variable NR_POINT_LIGHTS is set by preprocessor, and the command for this is generated by my applications code (Java). So when creating a shader program, I pass the desired start-amount of PintLights, complete the source text with the preprocessor command, compile, link and use. This works great.

Now I want to change this variable. I re-build the shader-source-string, re-compile and re-link a new shaderProgram and continue using this onoe. It just appears that all uniforms set in the old program are getting lost in the progress (of course, I once set them for the old program).

My ideas on how to fix this:

• Don't compile a new program, but rather somehow change the source data for the currently running shaders and somehow re-compile them, to continue using the program with the right uniform values
• Copy all uniform data from the old program to the newly generated one

What is the right way to do this? How do I do this? I'm not very experienced yet and don't know if any of my ideas is even possible.

Answer 1

You're looking for a Uniform Buffer or (4.3+ only) a Shader Storage Buffer.

struct Light {
    vec4 position;
    vec4 color;
    vec4 direction;
    /*Anything else you want*/
}

Uniform Buffer:

const int MAX_ARRAY_SIZE = /*65536 / sizeof(Light)*/;
layout(std140, binding = 0) uniform light_data {
    Light lights[MAX_ARRAY_SIZE];
};

uniform int num_of_lights;

Host Code for Uniform Buffer:

glGenBuffers(1, &light_ubo);
glBindBuffer(GL_UNIFORM_BUFFER, light_ubo);
glBufferData(GL_UNIFORM_BUFFER, sizeof(GLfloat) * static_light_data.size(), static_light_data.data(), GL_STATIC_DRAW); //Can be adjusted for your needs
GLuint light_index = glGetUniformBlockIndex(program_id, "light_data");
glBindBufferBase(GL_UNIFORM_BUFFER, 0, light_ubo);
glUniformBlockBinding(program_id, light_index, 0);
glUniform1i(glGetUniformLocation(program_id, "num_of_lights"), static_light_data.size() / 12); //My lights have 12 floats per light, so we divide by 12.

Shader Storage Buffer (4.3+ Only):

layout(std430, binding = 0) buffer light_data {
    Light lights[];
};

/*...*/

void main() {
    /*...*/
    int num_of_lights = lights.length();
    /*...*/
}

Host Code for Shader Storage Buffer (4.3+ Only):

glGenBuffers(1, &light_ssbo);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, light_ssbo);
glBufferData(GL_SHADER_STORAGE_BUFFER, sizeof(GLfloat) * static_light_data.size(), static_light_data.data(), GL_STATIC_DRAW); //Can be adjusted for your needs
light_ssbo_block_index = glGetProgramResourceIndex(program_id, GL_SHADER_STORAGE_BLOCK, "light_data");
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, light_ssbo);
glShaderStorageBlockBinding(program_id, light_ssbo_block_index, 0);

The main difference between the two is that Uniform Buffers:

• Have compatibility with older, OpenGL3.x hardware,
• Are limited on most systems to 65kb per buffer
• Arrays need to have their [maximum] size declared statically at the compile-time of the shader.

Whereas Shader Storage Buffers:

• Require hardware no older than 5 years
• Have a API mandated minimum allowable size of 16Mb (and most systems will allow up to 25% the total VRAM)
• Can dynamically query the size of any arrays stored in the buffer (though this can be buggy on older AMD systems)
• Can be slower than Uniform Buffers on the Shader side (roughly equivalent to a Texture Access)

Answer 2

Don't compile a new program, but rather somehow change the source data for the currently running shaders and somehow re-compile them, to continue using the program with the right uniform values

This isn't do-able at runtime if I'm understanding right (implying that you could change the shader-code of the compiled shader program) but if you modify the shader source text you can compile a new shader program. Thing is, how often do the number of lights change in your scene? Because this is a fairly expensive process to do.

You could specify a max number of lights if you don't mind having a limitation and only use the lights in the shader that have been populated with information, saving you the task of tweaking the source text and recompiling a whole new shader program, but that leaves you with a limitation on the number of lights (If you aren't planning on having absolutely loads of lights in your scene but are planning of having the number of lights change relatively often then this is probably going to be best for you)

However, if you really want to go down the route that you are looking at here:

Copy all uniform data from the old program to the newly generated one

You can look at using a Uniform Block. If you're going to be using shader programs with similar or shared uniforms, Uniform Blocks are a good way of managed those 'universal' uniform variables across your shade programs, or in your case the shader you are moving to as you grow the amount of lights in the shader. Theres a good tutorial on uniform blocks here

Lastly, depending on the OpenGL version you're using, you might still be able to achieve dynamic array sizes. OpenGL 4.3 introduced the ability to use buffers and have unbound array sizes, that you would use glBindBufferRange to send the length of your lights array to. You'll see more talk about that functionality in this question and this wiki reference.

The last would probably be my preference, but it depends on if you're aiming at hardware supporting older OpenGL versions.