So I was working on ConstPort recently and starting running into a weird error. It seems that sometimes when I call printf in my windows platform layer it freezes in the standard library code where it calls Windows' WriteFile function. The call to printf is happening inside the GlfwCursorEnteredCallback which is inside of the windowProc callback from windows. You can see the screenshot of visual studios for more information about the call-stack. Everything I am doing is on the MainThread.
After some more testing I've isolated the problem a bit. It seems it only happens when I have the console window in focus with some characters selected and then move my mouse over the application window to trigger the "Mouse entered window" printf call. The problem ONLY happens if text is selected in the console window.
This seems like a really weird problem to me so I wanted to ask here to see if anyone had any ideas about ways I might go about diagnosing the problem. Has anybody ever seen something like this before? Is it possible this is an actual bug with the Windows console window? Is there a way to find out what WriteFile is waiting on? Should I write a post on Microsoft's forums somewhere and see if I can get a response? Any suggestions on how to fix the problem? Since the bug is reproducible I can provide any other information upon request.
The console window is generated by windows when we have our main function defined as
In production mode we use
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow)
which doesn't generate a console window.
For those interested, the source for ConstPort can be found at https://bitbucket.org/stampede247/constport/src.
win32_main.cpp and plat_callbacks.cpp's GlfwCursorEnteredCallback are the main points of interest. The compiler and linker options can be found in build.bat. In debug mode I compile with /MTd /FC /Zi /EHsc /nologo /GS- /Gm- -GR- /EHa- /Fm /Od /Oi /WX /W4. The program is compiled by Visual Studio Community 2015's command line compiler in x64 mode. I'm running Windows 10 Pro Version 10.0.17134 Build 17134.