stb libraries as precompiled headers or statically linked

I spent the better portion of my afternoon yesterday attempting to reduce my compile times. I have 4 dlls that comprise an application, and each shares a header file that includes a few stb libraries. So when I recompile the whole project, that header file, along with all its contents are being needlessly compiled for each dll.

I tried to build the stb libraries as .obj, and pch, but after hours of attempts and searches I gave up the search.

Is it possible to achieve a compilation strategy where the stb libs are compiled once - and linked to each dll? Ideas, or resources on how to achieve this?
you make a single cpp file that #defines the *_IMPLEMENTATION and includes the header and compile it into an obj. Copy it around as needed and include it in the linker inputs.

In all other cpp files that include it you don't #define that symbol.
Could've sworn I tried that, but must've made a mistake. I'll try that again and see what happens.