I've just pushed the first bits of code!
high speed screen capture on osx
I'm on osx so pretty much all of this code is osx specific platform layer stuff. I've written some code to hook into an osx api that you can pass a callback function that gets called every time the screen is updated and lets you access the framebuffer. It's not the greatest api, since you pass it a callback which it calls on one of it's own threads so communicating the data back to my main program required a threadsafe buffer. I built one using atomics but I don't have much experience doing that so if anybody wants to take a look and give any feedback that would be great.
Another small optimization that seems to speed things up a lot is that I take advantage of the fact that the game has pixel art and compress the data to be 1 actual pixel per logical art pixel. I'm not sure if this will work in all cases but viewing the preview of the game the only downside seems to be choppy animation. This seriously lowers the amount of data I have to run through scene recognition stuff so should help make that fast.
Next step is doing some scene recognition!