Need suggestions with Windows hanging in printf/WriteFile

Hey everyone,

So I was working on ConstPort recently and starting running into a weird error. It seems that sometimes when I call printf in my windows platform layer it freezes in the standard library code where it calls Windows' WriteFile function. The call to printf is happening inside the GlfwCursorEnteredCallback which is inside of the windowProc callback from windows. You can see the screenshot of visual studios for more information about the call-stack. Everything I am doing is on the MainThread.



After some more testing I've isolated the problem a bit. It seems it only happens when I have the console window in focus with some characters selected and then move my mouse over the application window to trigger the "Mouse entered window" printf call. The problem ONLY happens if text is selected in the console window.



This seems like a really weird problem to me so I wanted to ask here to see if anyone had any ideas about ways I might go about diagnosing the problem. Has anybody ever seen something like this before? Is it possible this is an actual bug with the Windows console window? Is there a way to find out what WriteFile is waiting on? Should I write a post on Microsoft's forums somewhere and see if I can get a response? Any suggestions on how to fix the problem? Since the bug is reproducible I can provide any other information upon request.

The console window is generated by windows when we have our main function defined as
1
int main()

In production mode we use
1
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow)

which doesn't generate a console window.

For those interested, the source for ConstPort can be found at https://bitbucket.org/stampede247/constport/src.
win32_main.cpp and plat_callbacks.cpp's GlfwCursorEnteredCallback are the main points of interest. The compiler and linker options can be found in build.bat. In debug mode I compile with /MTd /FC /Zi /EHsc /nologo /GS- /Gm- -GR- /EHa- /Fm /Od /Oi /WX /W4. The program is compiled by Visual Studio Community 2015's command line compiler in x64 mode. I'm running Windows 10 Pro Version 10.0.17134 Build 17134.

Edited by Taylor Robbins on Reason: Initial post
Afaik if you select text in console, it stops receiving any output that is printed to it. It is effectively a way how to "pause" a program. This is expected behavior - it will "freeze" any console application that wants to produce output regardless of whether you have GUI, or compiler options used.

Edited by Mārtiņš Možeiko on
Ahh, that makes a lot of sense.

Alright so then a follow up question. Is there any good alternatives to the default windows console window to capture and display my stdout data? Preferably something that I can create (or not create) at run-time instead of compile-time switching out the entry point definition. I need something that runs on a separate thread and window so if the program freezes I can still scroll through the debug output. I know Visual Studio captures the output of the program but I'd like to be able to make a debug build that has a console window and doesn't have to be run in Visual Studio to see what is happening.
There's probably nothing ready like you want. You'll need to implement yourself. You could easily make your debug logs to go to queue, and then use separate thread to process this queue - then "freezing" console will only pause the thread, not whole application. Also you can always have only one entry point - WinMain one, and create console dynamically with AllocConsole.

But if its just for your own debugging then I recommend ConEmu. It is a replacement for cmd.exe and its great. I use it on all my Windows machines. I'm not really fond of its default settings (colors, fonts, tabs and status lines), but after a bit of configuring I like it a lot. The main things I use it - resizeable window, vertical select (which doesn't not freeze program output) and very good integration with FAR Manager (probably not relevant to you).
Awesome, thanks for the suggestions.

I know I had tried to look at making the Windows console appear at run-time with WinMain a while back and never found much success. I don't think I had found AllocConsole so I will look at how that behaves. That could solve one of my problems at least.

I'll have to take a look at ConEmu as well.