So I managed to implement raytracing (in Shadertoy). Gets expensive really fast but I'm quite pleased by the results. Rough materials are still too noisy, and I think the low sample count plays the major part unfortunately.
Sampling 4 times per sub pixel, sample count is halved for each bounce, 2x2 AA and 2 bounces. Last bounce always samples the environment light.