Hey all!
I'm writing a bunch of self-contained C/C++ headers for various tasks needed to make a game. The end goal is to actually have enough of these headers to cover what is needed for a 2D game. The plan is to write some tools for things like: string localization, loading/saving images, texture atlases, loading/playing audio, collision detection, a networking protocol, math library, OpenGL ES, loading tilemaps, and perhaps a couple other topics as time goes on. Oh, there are also a slew of smaller utility headers too!
Been pondering what to do for animations, tweening, and things of this sort. Perhaps also an A* header could really nice too! I've actually worked on an unfinished little Forest RTS, and in the source code (which is publicly available) there is a little header for A*, and one for animations. I will likely put out more professional versions of each in the future. I was considering finishing a lower level network protocol on top of UDP header. And finally, perhaps a sprite based font loading/rendering header with utf8 support.
Once enough of these headers are laid out they can be pulled together with some glue code to rapidly build up a game engine! I personally don't want to get into the area of making a game engine, but certainly am having a lot of fun making a very high-quality game framework. This is how I view tinyheaders.
The community here at handmade network is very friendly and likes this sort of content, so I'm sure it will be fun to post here as I get updates out over time :)
Cheers!
Randy
I'm writing a bunch of self-contained C/C++ headers for various tasks needed to make a game. The end goal is to actually have enough of these headers to cover what is needed for a 2D game. The plan is to write some tools for things like: string localization, loading/saving images, texture atlases, loading/playing audio, collision detection, a networking protocol, math library, OpenGL ES, loading tilemaps, and perhaps a couple other topics as time goes on. Oh, there are also a slew of smaller utility headers too!
Been pondering what to do for animations, tweening, and things of this sort. Perhaps also an A* header could really nice too! I've actually worked on an unfinished little Forest RTS, and in the source code (which is publicly available) there is a little header for A*, and one for animations. I will likely put out more professional versions of each in the future. I was considering finishing a lower level network protocol on top of UDP header. And finally, perhaps a sprite based font loading/rendering header with utf8 support.
Once enough of these headers are laid out they can be pulled together with some glue code to rapidly build up a game engine! I personally don't want to get into the area of making a game engine, but certainly am having a lot of fun making a very high-quality game framework. This is how I view tinyheaders.
The community here at handmade network is very friendly and likes this sort of content, so I'm sure it will be fun to post here as I get updates out over time :)
Cheers!
Randy
Edited by Randy Gaul
on
Newest addition is tinytiled.h. Here is a Reddit thread talking about the lib. The idea of the lib is to implement the minimum necessary to parse a JSON exported map file from the Tiled map editor (link). Tiled is a very high quality map editor and I really do recommend using it for your game if your game uses tiles.
tinytiled loads up and parses the JSON maps, and has features to let users explicitly control memory allocation, crt usage, and file handling. tinytiled has already been used (to my knowledge) by two other devs as a drop-in replacement for whatever they were using prior. They were kind enough to provide some really valuable feedback:
My main motivation for tinyspritebatch is to make hotloading files really trivial, that way I can hotload art and animations trivially without dealing with complicated texture atlases.
I actually started a thread on tigsource about tinyspritebatch here, with pretty pictures: https://forums.tigsource.com/index.php?topic=63197.0.
The overall design I have in mind, is a design to deal with file paths, file loading/unloading, hotloading of assets, and sprite batching all together. It took quite a while to come up with this design, and in the end I had quite a bit of help from Mattias Gustavsson (a member here on these forums). After discussing with him, I came up with this design:
In the meantime if anyone happens to come by with any thoughts or suggestions, please do let me know :)
tinytiled loads up and parses the JSON maps, and has features to let users explicitly control memory allocation, crt usage, and file handling. tinytiled has already been used (to my knowledge) by two other devs as a drop-in replacement for whatever they were using prior. They were kind enough to provide some really valuable feedback:
- https://www.reddit.com/r/gamedev/...on_map_parserloader_in_c/dweaq9t/
- https://github.com/RandyGaul/tinyheaders/issues/89
- https://github.com/RandyGaul/tinyheaders/issues/88
- https://github.com/RandyGaul/tinyheaders/issues/87
My main motivation for tinyspritebatch is to make hotloading files really trivial, that way I can hotload art and animations trivially without dealing with complicated texture atlases.
I actually started a thread on tigsource about tinyspritebatch here, with pretty pictures: https://forums.tigsource.com/index.php?topic=63197.0.
The overall design I have in mind, is a design to deal with file paths, file loading/unloading, hotloading of assets, and sprite batching all together. It took quite a while to come up with this design, and in the end I had quite a bit of help from Mattias Gustavsson (a member here on these forums). After discussing with him, I came up with this design:
- Use Mattias's assetsys.h for dealing with file paths, file loading/unloading, and for interfacing with zip archives (as if the archives were typical directories)
- tinyspritebatch.h deals with draw call optimization by hiding texture atlases behind a run-time API. This way assetsys, and the game itself, does not need to know about texture atlases whatsoever
- Create a header called tinyhotload.h, which wraps assetsys.h and provides hotload/hotswapping functionality. This is critical for a particular kind of development style that relies on super fast iteration time. Like images, audio, scripts, shaders, animations, tile levels, or any other kind of assets.
In the meantime if anyone happens to come by with any thoughts or suggestions, please do let me know :)
For basic 2D stuff, I do kinda similar things,
a basic atlas packer which repacks on each game start and then only filesystem events at runtime.
with simple API ~ loadImage(filename...), reloadImage, removeImage, createImage
and it does all the atlas stuff behind the scenes.
The sprite batching I do is simplest, most retarded thing, but I find it covers most cases, works on ancient openGL versions, has very few drawcalls, and is actually really fast.
Which basically is N preallocated vertex arrays, where each is identified by some key - which is a combination of (texture_id, blend_mode, layer) usually the key is very short like 8bits-10bits.
And all the sorting is done using Z buffer.
The only issue that I've run into so far is that the alpha blended things where order matter, have to be in the same texture_id (the same vertex array), so the atlas packer have to put them together in the same texture. With texture arrays this becomes basically a non-issue.
At some point you call flushBuffers() which loops through 2^N vertex buffers and sees if there's any vertices in them and then sets up the proper GL state (only when necessary) and uploads the relevant vertex data.
The beauty of this is, that the implementation is trivial, takes very little LoC, and has the simplest, dumbest API imaginable, like drawImage, drawImageRotated, drawText, drawLine, drawRect with some blendmode/alphatest options there. Which can be called in any order at any time without incurring any extra drawcalls. And it is fast.
The only downside is that you might take an extra hit on memory used from all the preallocated arrays, depending on how many bits your "render_key" has. But it is minor thing.
And flushBuffers() can be called multiple times within the frame if there's render to texture, or multiple viewports (as those can take many extra bits in render_key).
This can be alleviated if the buffers get resized as necessary (without copying if using the VirtualAlloc trick).
a basic atlas packer which repacks on each game start and then only filesystem events at runtime.
with simple API ~ loadImage(filename...), reloadImage, removeImage, createImage
and it does all the atlas stuff behind the scenes.
The sprite batching I do is simplest, most retarded thing, but I find it covers most cases, works on ancient openGL versions, has very few drawcalls, and is actually really fast.
Which basically is N preallocated vertex arrays, where each is identified by some key - which is a combination of (texture_id, blend_mode, layer) usually the key is very short like 8bits-10bits.
And all the sorting is done using Z buffer.
The only issue that I've run into so far is that the alpha blended things where order matter, have to be in the same texture_id (the same vertex array), so the atlas packer have to put them together in the same texture. With texture arrays this becomes basically a non-issue.
At some point you call flushBuffers() which loops through 2^N vertex buffers and sees if there's any vertices in them and then sets up the proper GL state (only when necessary) and uploads the relevant vertex data.
The beauty of this is, that the implementation is trivial, takes very little LoC, and has the simplest, dumbest API imaginable, like drawImage, drawImageRotated, drawText, drawLine, drawRect with some blendmode/alphatest options there. Which can be called in any order at any time without incurring any extra drawcalls. And it is fast.
The only downside is that you might take an extra hit on memory used from all the preallocated arrays, depending on how many bits your "render_key" has. But it is minor thing.
And flushBuffers() can be called multiple times within the frame if there's render to texture, or multiple viewports (as those can take many extra bits in render_key).
This can be alleviated if the buffers get resized as necessary (without copying if using the VirtualAlloc trick).
Edited by pragmatic_hero
on
Thanks for sharing. Sounds like that strategy might work pretty well for certain games. Did you try and see what happens when switching between levels, and thus drawing a bunch of different images over time? Also if premultiplied alpha is used, some of those blend modes can be coalesced.
Differences in tinyspritebatch from your design are:
Differences in tinyspritebatch from your design are:
- Rendering API agnostic (like OpenGL, GLES, DX, or others).
- Does not do preallocating. Instead atlases are formed at run-time based on which images are drawn at any given time. If images stop being drawn, they will eventually be popped out of atlases. As atlases start getting empty they are then merged together. When enough new images are being drawn they are placed into an atlas. The size of atlases (dimensions), and time-based parameters can all be tuned by the user.
- Sorting happens in CPU with quicksort.
- No file events. There is no concept of files at all as tinyspritebatch gets pixels in void* + stride format.
Edited by Randy Gaul
on
Actually no, which is a bug. This is because internally the quicksort is unstable. I'll have to modify it to be stable! Otherwise, yes the ordering is preserved :)
I'll fix this shortly when I get a chance.
I'll fix this shortly when I get a chance.
Edited by Randy Gaul
on
Randy Gaul
Thanks for sharing. Sounds like that strategy might work pretty well for certain games. Did you try and see what happens when switching between levels, and thus drawing a bunch of different images over time?
That works just fine, I certainly haven't thought about a case where there's more assets
than would fit into memory/vram. Fortunately majority of 2d games/indies simply do not have that amount of assets, so there's no need to evict unused things from the atlases. (except for file I/O doing removes and image resizing)
The fail case for relying on Zbuffer to do the sorting is when you have more than one blend mode (vertex array) where the order of rendering matters. Fortunately for the vast majority of time it is only ever a single order dependent blend-mode. And calling that extra flushBuffers() - if necessary - every once in a while is not that big of a deal.
The whole appeal of this approach is how retardedly simple and *fast* it is, while still covering a very good portion of what 2d game needs.
I haven't quite done any profiling, but my suspicion is yes, this kind of style can be very fast. Just using my header as an example, push functions can very much be just pushing a struct onto an array: https://github.com/RandyGaul/tiny...aster/tinyspritebatch.h#L818-L834
Next is the tick function. There are a couple loops jumping across linked lists. For now, for each atlas there is a bookkeeping struct allocated on the heap. Each of these structs contains a hash table holding more heap allocated arrays. This implies all performance here is dominated by cache misses. These cache misses could be avoided by changing the memory layout (all atlases in one array traversed linearly), and the same goes for the hash tables since the tables store key value pairs contiguously. But, it would be quite a bit of work, and in my opinion not really worth it. https://github.com/RandyGaul/tiny...aster/tinyspritebatch.h#L959-L978
Similarly the defrag function touches all the atlases and tables. This function is beefier, but all the big code paths early out when the drawn scene isn't changing rapidly. So most of the time this function is mostly a stub function since cost is amortized effectively. https://github.com/RandyGaul/tiny...ter/tinyspritebatch.h#L1453-L1622
Last is the flush function. Slightly more intensive than `spritebatch_push`, and in some cases may need to fetch pixels. In general, a negligibly fast function. The real overhead will certainly be in the user defined `batch_callback` function, since the user can unwittingly write a really slow function here. https://github.com/RandyGaul/tiny...ster/tinyspritebatch.h#L980-L1035
Next is the tick function. There are a couple loops jumping across linked lists. For now, for each atlas there is a bookkeeping struct allocated on the heap. Each of these structs contains a hash table holding more heap allocated arrays. This implies all performance here is dominated by cache misses. These cache misses could be avoided by changing the memory layout (all atlases in one array traversed linearly), and the same goes for the hash tables since the tables store key value pairs contiguously. But, it would be quite a bit of work, and in my opinion not really worth it. https://github.com/RandyGaul/tiny...aster/tinyspritebatch.h#L959-L978
Similarly the defrag function touches all the atlases and tables. This function is beefier, but all the big code paths early out when the drawn scene isn't changing rapidly. So most of the time this function is mostly a stub function since cost is amortized effectively. https://github.com/RandyGaul/tiny...ter/tinyspritebatch.h#L1453-L1622
Last is the flush function. Slightly more intensive than `spritebatch_push`, and in some cases may need to fetch pixels. In general, a negligibly fast function. The real overhead will certainly be in the user defined `batch_callback` function, since the user can unwittingly write a really slow function here. https://github.com/RandyGaul/tiny...ster/tinyspritebatch.h#L980-L1035
joe513
Does `spritebatch_push` draw the sprites in the same order that they are pushed in?
Now it does: https://github.com/RandyGaul/tiny...34778960ff462a1113fd630436c9f7018
Randy Gauljoe513
Does `spritebatch_push` draw the sprites in the same order that they are pushed in?
Now it does: https://github.com/RandyGaul/tiny...34778960ff462a1113fd630436c9f7018
That was quick.... I think this could be good for rendering isometric type games. The ability to sort and batch can be difficult to achieve.
Yes, it can work for any 2D sprite-based game :)
Finished first release of tinyspritebatch!
Overall it seems to work better than I hoped for, but for some reason I'm still a little skeptical it does not have any bugs or shortcomings. I don't have a real reason for feeling this way, other than that some of the code inside tinyspritebatch.h is more complicated than anticipated. Oh well, as time goes on it will feel more familiar.
Source: https://github.com/RandyGaul/tiny...ers/blob/master/tinyspritebatch.h
Demo from above gif via SDL and other similar headers: https://github.com/RandyGaul/tiny...and_tinytiled_and_tinyspritebatch
Next up is to hook up tile loading/batching/rendering with an asset hotloading header, and try to release the hotload header.
Overall it seems to work better than I hoped for, but for some reason I'm still a little skeptical it does not have any bugs or shortcomings. I don't have a real reason for feeling this way, other than that some of the code inside tinyspritebatch.h is more complicated than anticipated. Oh well, as time goes on it will feel more familiar.
Source: https://github.com/RandyGaul/tiny...ers/blob/master/tinyspritebatch.h
Demo from above gif via SDL and other similar headers: https://github.com/RandyGaul/tiny...and_tinytiled_and_tinyspritebatch
Next up is to hook up tile loading/batching/rendering with an asset hotloading header, and try to release the hotload header.
Got hotload header finished up! Decided to name it tinyfilewatch.h. https://github.com/RandyGaul/tinyheaders/blob/master/tinyfilewatch.h
Ended up being fairly easy to implement, but it embeds three other headers which is kind of a lot. It embeds tinyfiles.h (for cross-platform directory traversals), strpool.h by Mattias Gustavsoon for string interning, and hashtable.h by Mattias Gustavsson.
All in all I'm really pleased with how it turned out! Very useful and solid header.
Now to hook it up to tinytiled, tinygl, and tinyspritebatch to make a nice demo. Maybe I will even hook it up to tinyc2 and make a 2D platformer demo. The cool part of the demo will be hotloading of images/maps :)
Ended up being fairly easy to implement, but it embeds three other headers which is kind of a lot. It embeds tinyfiles.h (for cross-platform directory traversals), strpool.h by Mattias Gustavsoon for string interning, and hashtable.h by Mattias Gustavsson.
All in all I'm really pleased with how it turned out! Very useful and solid header.
Now to hook it up to tinytiled, tinygl, and tinyspritebatch to make a nice demo. Maybe I will even hook it up to tinyc2 and make a 2D platformer demo. The cool part of the demo will be hotloading of images/maps :)
Edited by Randy Gaul
on