handmade.network » Forums » Hot Swap your shaders!
MandleBro
Jack Mott
96 posts
1 project

Web Developer by day, game hobbyist by night. Fond of C and F#

#10786 Hot Swap your shaders!
2 months, 3 weeks ago Edited by Jack Mott on Feb. 6, 2017, 10:39 p.m.

While watching some of the JBlow streams this past week he mentioned how he had set up hot swappable shaders. Where you can edit the shader file while the game is running, and as soon as you save it swaps in the new one so you can see the results.

Such an easy feature to add. Poll the file system for updated shaders, load in the new one when it changes, wrap it with some #ifdef DEBUG constructs and it won't even affect runtime at all, and saves a ton of dev time.

I got it up and running this weekend with monogame:

CaptainKraft
Jeremiah
120 posts
2 projects

Father, husband, C programmer, and Linux apologist. Think before you code.

#10791 Hot Swap your shaders!
2 months, 3 weeks ago

Pretty damn cool. It seems crucial to have during development.

Build a man a fire, he'll be warm for a day.
Set a man on fire, he'll be warm forever.
Chen96
Chen
10 posts
#10812 Hot Swap your shaders!
2 months, 3 weeks ago

Yo this is pretty cool. You know how to do uber shaders? like vertex and fragment all in one file and compile them with different macros?

Alex
mmozeiko
Mārtiņš Možeiko
1281 posts
1 project
#10813 Hot Swap your shaders!
2 months, 3 weeks ago Edited by Mārtiņš Možeiko on Feb. 8, 2017, 6:49 a.m.

Ubershaders is not for putting vertex and fragment shaders in one. That's not possible at all with GL or D3D. They require different source for each.

Ubershaders are for combining different materials or effects in one. Like some objects maybe have specular texture, but some not, then you can do something like this:
1
2
3
4
5
vec4 spec = vec4(0);
if (uHasSpecular)
{
    vec4 = tex2D(specularSampler, uv);
}

Then you set uHasSpecular unfirom to true or false for specific objects (or shader programs). Theoretically driver should notice that this uniform doesn't change and compile only one variant (or two different ones for two boolean values).


If you want to just put vertex and fragment shader in same source *file*, then you can easily do that with preprocessor:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
.. // put here common part, like uniforms and varyings
#if defined(VERTEX_SHADER)
void main() {
  gl_Position = ...;
}
#elif defined(FRAGMENT_SHADER)
void main() {
  gl_Color = ...;
}
#endif

And then simply prepend "#define VERTEX_SHADER\n" string when compiling this source code for vertex shader object, and "#define FRAGMENT_SHADER\n" when compiling fragment shader. That's pretty much it.

With glShaderSource you can do this even without manually concatenating strings - it allows to pass multiple strings that are processed sequentially. So something like this will work just fine:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
const char* vertex_src[] = {
    "#define VERTEX_SHADER\n",
    combined_source_code,
};
const char* fragment_src[] = {
    "#define FRAGMENT_SHADER\n",
    combined_source_code, // same string as in previous vertex_src array
};
glShaderSource(vertex_object, ArrayCount(vertex_src), vertex_src, NULL);
glShaderSource(fragment_object, ArrayCount(fragment_src), fragment_src, NULL);
ratchetfreak
251 posts
#10815 Hot Swap your shaders!
2 months, 2 weeks ago

mmozeiko:
Ubershaders is not for putting vertex and fragment shaders in one. That's not possible at all with GL or D3D. They require different source for each.


Actually in D3D and vulkan you have to specify the name of the entry point so you can put the vertex and fragment shaders together in a single file. In opengl it is always main.
mmozeiko
Mārtiņš Možeiko
1281 posts
1 project
#10816 Hot Swap your shaders!
2 months, 2 weeks ago

Right. I explained that poorly. What I wanted to say is that runtime will treat both shaders as completely objects. Yes, they may com from same source text, but compiler will process source code separately when compiling to bytecode / gpu asm - like my example with OpenGL above.
Floresy
Tom
21 posts

None

#10820 Hot Swap your shaders!
2 months, 2 weeks ago

a pitfall to be aware of, for others attempting this: check if the new shader source compiles correctly before throwing away the old shader objects. Otherwise you have my implementation where reflexively saving the source file before you're actually done editing will crash your game :P

0xdedededede.....
ratchetfreak
251 posts
#10821 Hot Swap your shaders!
2 months, 2 weeks ago

And if you are gong to be doing a lot of hot swapping, be aware of how to clean up old programs

For a typical game with a limited amount of programs created that's not an issue but when hot swapping you don't really want to keep a dozen unused programs around confusing the driver's cache.
Floresy
Tom
21 posts

None

#10822 Hot Swap your shaders!
2 months, 2 weeks ago

at least with opengl, if you're detaching your shader and shaderprogram objects asap, shouldn't that be enough the let the driver clean it up?

0xdedededede.....
ratchetfreak
251 posts
#10823 Hot Swap your shaders!
2 months, 2 weeks ago

Floresy:
at least with opengl, if you're detaching your shader and shaderprogram objects asap, shouldn't that be enough the let the driver clean it up?


Detaching (using another program) doesn't destroy the program. You need to call glDeleteProgram for that. Same with the shader object attached to it.
MandleBro
Jack Mott
96 posts
1 project

Web Developer by day, game hobbyist by night. Fond of C and F#

#10833 Hot Swap your shaders!
2 months, 2 weeks ago

Floresy:
a pitfall to be aware of, for others attempting this: check if the new shader source compiles correctly before throwing away the old shader objects. Otherwise you have my implementation where reflexively saving the source file before you're actually done editing will crash your game :P


good point!
In my case I would just end up reloading the old one, but that is also bad. Things would not change and I won't know why.
So I'll need to detect the failed compilation and report it.
Mebourne
Scott Hunt
21 posts

Father, Thinker, Mechanical Engineer @NASA, and C/C++ Hobby Enthusiast / @TexxStudio

#10835 Hot Swap your shaders!
2 months, 2 weeks ago

Awesome, looks great.

Just got this up and running in my engine as well on Monday. Very simple with a quick evaluation of the last write time of the files. As noted here and what Martin's recommendation helped point out in the Handmade Hero forums; validate and verify all OpenGL calls. I thought I was linking compiled shaders successfully, as the compile wasn't complaining, but it ended up having link errors due to an existing main() method being linked to the old Program Id. You must delete the old program and can not attempt to just recompile over top the Id.

None
gbluma
Garrett Bluma
1 posts
#10851 Hot Swap your shaders!
2 months, 2 weeks ago

This is cool stuff--I think I might try to add something like it to my own project. :)

For those who are just trying to debug a tricky problem though, and don't want to make life harder, I should mention you can do something similar in RenderDoc. You can actually capture a frame of your OpenGL program (unmodified), open up the framebuffer, edit your shader, and see what would happen right there.

There's a good walkthrough of it:



The advantage of writing your own is that you can still "use" your program and the changes persist.
mmozeiko
Mārtiņš Možeiko
1281 posts
1 project
#10856 Hot Swap your shaders!
2 months, 2 weeks ago

RenderDoc is great, but its only for core context.
But there are other good tools for OpenGL:
- ApiTrace
- Intel Graphics Performance Analyzers
- AMD CodeXL
- NVIDIA Nsight