&meeseengine

This past month has been all about integrating the game with my internal dev tools and revamping core engine features. The biggest milestone has been completely overhauling concurrent heap defragmentation to achieve ultra-low latency performance and O(1) scalability across all the platforms. On average and worst-case, pause times are now down to microsecond levels on all platforms.

The new defragmentation algorithm is something I developed from first principles. I've read various research papers, but I couldn't find anything that matched what I needed, so I ended up inventing my own solution. Heap defragmentation can also function as a garbage collector, but I retained the malloc/free semantics as a flexible guideline for the memory manager to either use or ignore. This approach also allows for benchmarking my solution against malloc/free and arenas.

Heap defragmentation is critical for making this type of game on the Gamecube and Dreamcast ports a reality. These consoles are severely limited in memory, where running out of usable memory is a real risk when address space gets tight. The recent improvements ensure that heap defragmentation will not cause stutters, even as the number of allocated objects, such as chunks and entities, grows into the millions.

The first clip shows the PC version running with 100,000 objects (top-left of video) at a render distance of 32, handling defragmentation effortlessly. In the second clip, I pushed it up to half a million objects with a render distance of 64, and it didn't even flinch — the pause times stayed rock solid.