Handmade Network»Forums
Jack Mott
110 posts
Web Developer by day, game hobbyist by night.
Hot Swap your shaders!
Edited by Jack Mott on
While watching some of the JBlow streams this past week he mentioned how he had set up hot swappable shaders. Where you can edit the shader file while the game is running, and as soon as you save it swaps in the new one so you can see the results.

Such an easy feature to add. Poll the file system for updated shaders, load in the new one when it changes, wrap it with some #ifdef DEBUG constructs and it won't even affect runtime at all, and saves a ton of dev time.

I got it up and running this weekend with monogame:

Jeremiah Goerdt
209 posts / 2 projects
Programmer, Linux apologist, and not-so-wiseman.
Hot Swap your shaders!
Pretty damn cool. It seems crucial to have during development.
Chen
102 posts / 2 projects
Some guy trying to make some video game.
Hot Swap your shaders!
Yo this is pretty cool. You know how to do uber shaders? like vertex and fragment all in one file and compile them with different macros?
Mārtiņš Možeiko
2237 posts / 1 project
Hot Swap your shaders!
Edited by Mārtiņš Možeiko on
Ubershaders is not for putting vertex and fragment shaders in one. That's not possible at all with GL or D3D. They require different source for each.

Ubershaders are for combining different materials or effects in one. Like some objects maybe have specular texture, but some not, then you can do something like this:
1
2
3
4
5
vec4 spec = vec4(0);
if (uHasSpecular)
{
    vec4 = tex2D(specularSampler, uv);
}

Then you set uHasSpecular unfirom to true or false for specific objects (or shader programs). Theoretically driver should notice that this uniform doesn't change and compile only one variant (or two different ones for two boolean values).


If you want to just put vertex and fragment shader in same source *file*, then you can easily do that with preprocessor:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
.. // put here common part, like uniforms and varyings
#if defined(VERTEX_SHADER)
void main() {
  gl_Position = ...;
}
#elif defined(FRAGMENT_SHADER)
void main() {
  gl_Color = ...;
}
#endif

And then simply prepend "#define VERTEX_SHADER\n" string when compiling this source code for vertex shader object, and "#define FRAGMENT_SHADER\n" when compiling fragment shader. That's pretty much it.

With glShaderSource you can do this even without manually concatenating strings - it allows to pass multiple strings that are processed sequentially. So something like this will work just fine:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
const char* vertex_src[] = {
    "#define VERTEX_SHADER\n",
    combined_source_code,
};
const char* fragment_src[] = {
    "#define FRAGMENT_SHADER\n",
    combined_source_code, // same string as in previous vertex_src array
};
glShaderSource(vertex_object, ArrayCount(vertex_src), vertex_src, NULL);
glShaderSource(fragment_object, ArrayCount(fragment_src), fragment_src, NULL);
498 posts
Hot Swap your shaders!
mmozeiko
Ubershaders is not for putting vertex and fragment shaders in one. That's not possible at all with GL or D3D. They require different source for each.


Actually in D3D and vulkan you have to specify the name of the entry point so you can put the vertex and fragment shaders together in a single file. In opengl it is always main.
Mārtiņš Možeiko
2237 posts / 1 project
Hot Swap your shaders!
Right. I explained that poorly. What I wanted to say is that runtime will treat both shaders as completely objects. Yes, they may com from same source text, but compiler will process source code separately when compiling to bytecode / gpu asm - like my example with OpenGL above.
Tom
25 posts
None
Hot Swap your shaders!
a pitfall to be aware of, for others attempting this: check if the new shader source compiles correctly before throwing away the old shader objects. Otherwise you have my implementation where reflexively saving the source file before you're actually done editing will crash your game :P
498 posts
Hot Swap your shaders!
And if you are gong to be doing a lot of hot swapping, be aware of how to clean up old programs

For a typical game with a limited amount of programs created that's not an issue but when hot swapping you don't really want to keep a dozen unused programs around confusing the driver's cache.
Tom
25 posts
None
Hot Swap your shaders!
at least with opengl, if you're detaching your shader and shaderprogram objects asap, shouldn't that be enough the let the driver clean it up?
498 posts
Hot Swap your shaders!
Floresy
at least with opengl, if you're detaching your shader and shaderprogram objects asap, shouldn't that be enough the let the driver clean it up?


Detaching (using another program) doesn't destroy the program. You need to call glDeleteProgram for that. Same with the shader object attached to it.
Jack Mott
110 posts
Web Developer by day, game hobbyist by night.
Hot Swap your shaders!
Floresy
a pitfall to be aware of, for others attempting this: check if the new shader source compiles correctly before throwing away the old shader objects. Otherwise you have my implementation where reflexively saving the source file before you're actually done editing will crash your game :P


good point!
In my case I would just end up reloading the old one, but that is also bad. Things would not change and I won't know why.
So I'll need to detect the failed compilation and report it.
Scott Hunt
41 posts
Father, Thinker, Mechanical Engineer @NASA, and C/C++ Hobby Enthusiast / @TexxStudio
Hot Swap your shaders!
Awesome, looks great.

Just got this up and running in my engine as well on Monday. Very simple with a quick evaluation of the last write time of the files. As noted here and what Martin's recommendation helped point out in the Handmade Hero forums; validate and verify all OpenGL calls. I thought I was linking compiled shaders successfully, as the compile wasn't complaining, but it ended up having link errors due to an existing main() method being linked to the old Program Id. You must delete the old program and can not attempt to just recompile over top the Id.
Garrett Bluma
1 posts
Hot Swap your shaders!
This is cool stuff--I think I might try to add something like it to my own project. :)

For those who are just trying to debug a tricky problem though, and don't want to make life harder, I should mention you can do something similar in RenderDoc. You can actually capture a frame of your OpenGL program (unmodified), open up the framebuffer, edit your shader, and see what would happen right there.

There's a good walkthrough of it:



The advantage of writing your own is that you can still "use" your program and the changes persist.
Mārtiņš Možeiko
2237 posts / 1 project
Hot Swap your shaders!
RenderDoc is great, but its only for core context.
But there are other good tools for OpenGL:
- ApiTrace
- Intel Graphics Performance Analyzers
- AMD CodeXL
- NVIDIA Nsight