I'm trying to write a windows library that needs to respond to specific window messages (more specifically WM_COMMAND). I would like to not do it inside the window procedure, since that will be written by the user of the library and it would be nice to not have to force them to do something in there.

The problem is that WM_COMMANDs bypass the message queue and are sent directly to the window proc, so I can't, say, examine the queue with GetQueueStatus and filter out the messages I don't care about.

I found that windows offers the SetWindowsHookExA function, but that only works if called from a separate dll, and since mine is a std-like library, that's not going to work.

Does anybody have any idea how to do what I'm trying to do, possibily without making things cumbersome from the user side of the code? Is that even possible?

Could you maybe hook the window procedure with SetWindowLongPtr (GWLP_WNDPROC) and then call the original procedure with CallWindowProc?

I found that windows offers the SetWindowsHookExA function, but that only works if called from a separate dll, and since mine is a std-like library, that's not going to work.

Reading the doc of SetWindowsHookExA, it seems to me that if you specify the thread id you can call code that is in the exe. I haven't tested it but something like:

SetWindowsHookExA( WH_CALLWNDPROC, hook_function, 0, GetCurrentThreadId() );

I don't know what your library is doing, but as a user I think I would prefer having to add a call to the lib function in WM_COMMAND myself instead of having a hook because then I'm aware that some code is being called and I can choose not to call it (when debugging something it's nice to be able to make sure nothing unexpected is happening).

That's what I prefer as user of library too - no hidden magic or hooks. Let me call functions explicitly, which allows me to arrange the code the best way myself, instead of guessing what would library prefer..

Like in my PS4/5 controller "library" using RawInput: https://gist.github.com/mmozeiko/c136c1cfce9fe4267f3c8f7b90f8e4d4

Instead of hooking any WindowProc, I just ask user to call ps_init/ps_update/ps_input from relevant messages.