I decided to port the probe based lighting stuff to the GPU using D3D11... Finally starting to get some decent results!
Things got too slow in the software renderer and I didn't feel like doing multithreading etc.. But it was a very useful reference for getting stuff running on the GPU.