Great idea to have a WIP section! Here's mine:
Pixie Game System is a (yet another?) C++ 2D game engine for making retro-style games using pixel/palette art. It is actually a rework/rewrite of my old game engine which was written 10 years ago. You learn a lot in 10 years, and I figured it was time to put that into practice. Probably the biggest change is that I am writing all the core systems of the engine as single-header-only libs (in the style of stb_image), with the actual "engine lib" just being another header-only lib which ties them together and expose the functionality in a nicer way. This way, each core system can be used stand-alone as well - useful for people who have their own engines and just wants to grab a small part of Pixie. The code is released as public domain (with the option to use MIT for areas where PD is problematic).
At the moment, I have a lot of the core system libs in place, and I'm working on implementing "pixie.hpp", the user facing layer providing a consistent interface. There's a few hundred TODO:s in there, and probably a couple of good-sized core systems still to implement (or rather re-implement based on the original engine code), so it is probably still some months before a full release - I'm hoping to release before the end of the year. And if there's any interest, I don't mind doing a preview release earlier.
The various supporting libs have different API styles, some are C, some are C++, depending on what I found most suitable. The main engine API though, is aiming for consistency and simplicity, and takes a lot of inspiration from old school BASIC languages when it comes to feature set and naming. Here's a simple, complete example drawing some lines:
And this is what it looks like when you run it (the example uses the optional CRT emulation shader, to get that "really blurry old TV" look...)
Pixie Game System is a (yet another?) C++ 2D game engine for making retro-style games using pixel/palette art. It is actually a rework/rewrite of my old game engine which was written 10 years ago. You learn a lot in 10 years, and I figured it was time to put that into practice. Probably the biggest change is that I am writing all the core systems of the engine as single-header-only libs (in the style of stb_image), with the actual "engine lib" just being another header-only lib which ties them together and expose the functionality in a nicer way. This way, each core system can be used stand-alone as well - useful for people who have their own engines and just wants to grab a small part of Pixie. The code is released as public domain (with the option to use MIT for areas where PD is problematic).
At the moment, I have a lot of the core system libs in place, and I'm working on implementing "pixie.hpp", the user facing layer providing a consistent interface. There's a few hundred TODO:s in there, and probably a couple of good-sized core systems still to implement (or rather re-implement based on the original engine code), so it is probably still some months before a full release - I'm hoping to release before the end of the year. And if there's any interest, I don't mind doing a preview release earlier.
The various supporting libs have different API styles, some are C, some are C++, depending on what I found most suitable. The main engine API though, is aiming for consistency and simplicity, and takes a lot of inspiration from old school BASIC languages when it comes to feature set and naming. Here's a simple, complete example drawing some lines:
And this is what it looks like when you run it (the example uses the optional CRT emulation shader, to get that "really blurry old TV" look...)
Edited by @Mattias_G
on
Thanks :) It can be seen more clearly in a couple of small games I have here: www.mattiasgustavsson.com (made with the old engine, but the same effect). I intend to modify the implementation to allow for customized settings, as I think it is too pronounced for some cases, and just right for others.
As for my code color scheme, I think the word you are looking for is "classic" :D (go Borland!)
As for my code color scheme, I think the word you are looking for is "classic" :D (go Borland!)
So, I wanted to share a little bit about the background of this project. If I had a blog, I guess I'd put this there. But this place will do just as nicely, I think.
I have been a games programmer for a long time - wrote my first little game in 1983, you know the story, it's a common one. Got a job as a programmer in the games industry in 1999, and like many of my colleagues at the time, I was messing around with writing my own 3D engine in my spare time (remember, this was before Unity, when an engine license would cost you an arm and a leg). And just like many of the others, I never really got to the point of actually building a game on it - it was just a constant messing around with the tech.
Fast forward to 2007 - I was working for an independent mid-size studio, and we got a small contract to make a game for the, then booming, casual games market - the online portals, like bigfish etc. This was to be a simple 2D game, so we were given this in-house engine made by the publisher. It... didn't impress. It was slow, bloated, used a ton of middleware that didn't exactly gel in a natural way. So I started thinking that there must be a better way to do this. Not for the game at hand, that was on too tight a schedule, but just generally solving it better.
So while I was working on that game, I put my hobby 3D engine aside, and started working on a new 2D engine specifically tailored to small/simple games, where speed of development was important, but not at the expense of basic efficiency. I also made a point to not add any feature unless I was implementing a game (or prototype) that actually needed it, as I didn't want to end up with something big and bloated.
I quickly realized that this was a lot more fun than the old 3D engine I had been working on, in different forms, over the years. So I spent more time on this, but also more focus on actually making games with it, taking part in some game jams and such. I made a bunch of little game embryos and experiments, and kept growing the engine as needed. I called it Pixie, as it was meant to be small, lightweight and cute. I released all the source code as public domain from the start - I just wanted to use this engine for my games, not own the code.
As with any monolithic code base where you grow it over time, it gets a bit rigid after a while. It's not as easy to change something without affecting, or breaking, something else. Not knowing a better way at the time, I had implemented it as straight up by-the-book OOP - interfaces, inheritance, the whole lot. Which I still think is ok for certain parts of a system, if used responsibly, especially in the high-level gameplay code. But it is not really helpful to religiously carry it all the way down into the lowest, platform-interfacing layers of the engine, like I had done.
One thing that I found particularly annoying, was that whenever I wanted to work on something which didn't align with the Pixie way of doing things, there were very few options available to me - I either had to adapt the project to fit engine restrictions, or reimplement a lot of the stuff I had already implemented for Pixie. Or, use some other library/framework altogether. Every part of Pixie had dependencies on every other part, and there was no way to re-use just part of the code for a stand-alone project. I'm exaggerating a bit - it wasn't all *that* bad. It wasn't like I hadn't tried to be sensible about dependencies and such - but it sure felt like that most of the time.
Sometime around 2011-2012, while I was still happily making little games in Pixie, I had started to toy with the idea of doing an extensive refactoring, to make it a lot more modular, and also to move some parts of it away from OOP. Around this time, data oriented design was making waves in the industry, and this made me re-think my coding style on many levels, and got me questioning a lot of the commonly accepted "facts" about OOP. A vision started to form, on how I wanted to rework my old Pixie codebase into something more current to my evolved tastes.
Then at the start of 2013, I ended up working for one of the big publishers, for one of their big game studios in Sweden. And for the first time, my contract had an incredibly strict non-compete clause which left no room for doing *anything* game or programming related in your spare time. I lasted about a year and a half at that place - and I am surprised I lasted that long, for many reasons. But a big one was that I missed my hobby - making little games and working on my engine. So I finally left the games industry, after 15 years, and got a "real" programming job instead.
First thing I did with my new-found freedom, was start making a game again (http://gamejolt.com/games/extrication/62070). I found the ShareCart1000 game jam (a running jam, (http://www.sharecart1000.com) which had a couple of restrictions - one was a very limited save game format, and the other was a color palette of only 16 fixed colors. Not having made a palettized game in years (decades even), I was surprised by how much fun I had with it, and realized that this was what I wanted to do more of - palette-based retro-style games.
So, in 2015 I started to, little by little, refactor my engine. To streamline everything, make it as modular as possible (or at least practical). By now, I had found the wonderful stb libraries (https://github.com/nothings/stb), and there are many things to like (and mimic) about them. One is the distribution aspect. With a single header library, there's only one file to move around - declarations, implementations and documentation is all rolled into one. Another thing is build configuration. No more wasted time trying to figure out how to set up make files or generate projects to build a lib. You just include it and go. A third benefit is configuration - being able to modify the librarys behavior through #defines (eg. giving a custom allocation function).
The plan I've been working to since, is to refactor all the core components of my Pixie engine into stand-alone stb-style single header libraries, and then write one "pixie.h" single header library containing glue-code and helper code to tie all my libs together into an engine/framework of similar ease-of-use as my old Pixie engine.
Over the next few posts, I will present some of the core libraries that has come out of this process, in the hope that they might be useful for others as well. The pixie.h engine lib itself, which we will get to eventually, will perhaps not be as useful, as I have intentionally made it very tailored to the games I want to make - low-res, palettized retro styled games. I have, however, kept the other libraries more generic.
I have been a games programmer for a long time - wrote my first little game in 1983, you know the story, it's a common one. Got a job as a programmer in the games industry in 1999, and like many of my colleagues at the time, I was messing around with writing my own 3D engine in my spare time (remember, this was before Unity, when an engine license would cost you an arm and a leg). And just like many of the others, I never really got to the point of actually building a game on it - it was just a constant messing around with the tech.
Fast forward to 2007 - I was working for an independent mid-size studio, and we got a small contract to make a game for the, then booming, casual games market - the online portals, like bigfish etc. This was to be a simple 2D game, so we were given this in-house engine made by the publisher. It... didn't impress. It was slow, bloated, used a ton of middleware that didn't exactly gel in a natural way. So I started thinking that there must be a better way to do this. Not for the game at hand, that was on too tight a schedule, but just generally solving it better.
So while I was working on that game, I put my hobby 3D engine aside, and started working on a new 2D engine specifically tailored to small/simple games, where speed of development was important, but not at the expense of basic efficiency. I also made a point to not add any feature unless I was implementing a game (or prototype) that actually needed it, as I didn't want to end up with something big and bloated.
I quickly realized that this was a lot more fun than the old 3D engine I had been working on, in different forms, over the years. So I spent more time on this, but also more focus on actually making games with it, taking part in some game jams and such. I made a bunch of little game embryos and experiments, and kept growing the engine as needed. I called it Pixie, as it was meant to be small, lightweight and cute. I released all the source code as public domain from the start - I just wanted to use this engine for my games, not own the code.
As with any monolithic code base where you grow it over time, it gets a bit rigid after a while. It's not as easy to change something without affecting, or breaking, something else. Not knowing a better way at the time, I had implemented it as straight up by-the-book OOP - interfaces, inheritance, the whole lot. Which I still think is ok for certain parts of a system, if used responsibly, especially in the high-level gameplay code. But it is not really helpful to religiously carry it all the way down into the lowest, platform-interfacing layers of the engine, like I had done.
One thing that I found particularly annoying, was that whenever I wanted to work on something which didn't align with the Pixie way of doing things, there were very few options available to me - I either had to adapt the project to fit engine restrictions, or reimplement a lot of the stuff I had already implemented for Pixie. Or, use some other library/framework altogether. Every part of Pixie had dependencies on every other part, and there was no way to re-use just part of the code for a stand-alone project. I'm exaggerating a bit - it wasn't all *that* bad. It wasn't like I hadn't tried to be sensible about dependencies and such - but it sure felt like that most of the time.
Sometime around 2011-2012, while I was still happily making little games in Pixie, I had started to toy with the idea of doing an extensive refactoring, to make it a lot more modular, and also to move some parts of it away from OOP. Around this time, data oriented design was making waves in the industry, and this made me re-think my coding style on many levels, and got me questioning a lot of the commonly accepted "facts" about OOP. A vision started to form, on how I wanted to rework my old Pixie codebase into something more current to my evolved tastes.
Then at the start of 2013, I ended up working for one of the big publishers, for one of their big game studios in Sweden. And for the first time, my contract had an incredibly strict non-compete clause which left no room for doing *anything* game or programming related in your spare time. I lasted about a year and a half at that place - and I am surprised I lasted that long, for many reasons. But a big one was that I missed my hobby - making little games and working on my engine. So I finally left the games industry, after 15 years, and got a "real" programming job instead.
First thing I did with my new-found freedom, was start making a game again (http://gamejolt.com/games/extrication/62070). I found the ShareCart1000 game jam (a running jam, (http://www.sharecart1000.com) which had a couple of restrictions - one was a very limited save game format, and the other was a color palette of only 16 fixed colors. Not having made a palettized game in years (decades even), I was surprised by how much fun I had with it, and realized that this was what I wanted to do more of - palette-based retro-style games.
So, in 2015 I started to, little by little, refactor my engine. To streamline everything, make it as modular as possible (or at least practical). By now, I had found the wonderful stb libraries (https://github.com/nothings/stb), and there are many things to like (and mimic) about them. One is the distribution aspect. With a single header library, there's only one file to move around - declarations, implementations and documentation is all rolled into one. Another thing is build configuration. No more wasted time trying to figure out how to set up make files or generate projects to build a lib. You just include it and go. A third benefit is configuration - being able to modify the librarys behavior through #defines (eg. giving a custom allocation function).
The plan I've been working to since, is to refactor all the core components of my Pixie engine into stand-alone stb-style single header libraries, and then write one "pixie.h" single header library containing glue-code and helper code to tie all my libs together into an engine/framework of similar ease-of-use as my old Pixie engine.
Over the next few posts, I will present some of the core libraries that has come out of this process, in the hope that they might be useful for others as well. The pixie.h engine lib itself, which we will get to eventually, will perhaps not be as useful, as I have intentionally made it very tailored to the games I want to make - low-res, palettized retro styled games. I have, however, kept the other libraries more generic.
So, as I promised, I want to start showing the core libraries of the engine. I am not releasing libs until they are reasonably complete, at least a little bit tested, and somewhat well documented, and it is taking me longer than I had anticipated to get there - hopefully I will pick up speed the further I get :)
As I have mentioned above, one of my goals with this rewrite have been to make sure each core part of the engine can be used as a standalone library. First out, is a library I call app.h, which can be found here:
https://github.com/mattiasgustavsson/libs/blob/master/app.h
app.h is the basic "platform layer" for my engine. It handles creating a window, reading input, displaying pixels and playing sound samples. I guess you could say it is doing the things one would normally use something like SDL for, but only the subset which is actually platform dependent - all the "nice to have" things like loading and rendering bitmaps, drawing fonts, loading soundfiles etc, are intentionally left out.
This is probably worth talking a bit about, as it is somewhat central to the way I approach development. When writing a library, or a module for a program, there are different things that can be prioritized. I think it is essential to have the priorities clear before you start writing the code. This is not to say that you need to know what code to write - I am a big believer in explorative programming. But you need to know the priorities. I think it is too common that programmers default to prioritizing ease-of-use above everything else, and for all the code they write. But there are other factors that can be more important, depending on circumstances. For example, sometimes performance is more important than ease-of-use. Sometimes control is at the top, maximizing the level of control the user have over the libraries functionality. Or, as is the case with the platform layer, ease of porting can be the top priority.
As can be seen in app.h, the API is very minimalistic. At various points of development, it used to have a lot more functionality, many more nice to haves. But all that would give me here, is more code to port to other platforms.
A much better approach, is to keep this library very small and minimalistic, and then build other libraries that can provide ease-of-use, which can run on top of this. The challenge is to minimize dependencies between libs. I don't want the higher level libraries to have direct dependencies on app.h - instead, I want to structure it so that when I write a program, I can pull in app.h and some other libraries and write some simple "glue code" to connect them. Keeping libraries independent like that makes it much easier to re-use them for different purposes. If I write, for example, a font renderer, I would want to be able to plug it into either app.h or some other system, like maybe SDL, depending on what I am making.
We will see later how to create other libraries on top of app.h. But even though I have libs to make it easier to use, I often use app.h directly for many smaller projects, where what I'm doing is more specialized, and where I don't particularly need the ease of use of the higher level libs. In return, I get a smaller code base and less overhead.
The library itself is a single-file header-only library like Sean Barrets stb libraries. A basic example of its use could look like this:
The first line is the typical header-only define to get the implementation. The next line, defining APP_WINDOWS, selects the platform we are running on. At the moment, only windows is supported, but I'll get around to porting it to Linux and Mac too at some point.
The action starts at line 32, in the entry point of the program, standard "main" function. We simply call app_run, and pass in a function pointer to our custom "app_proc". The app_proc is simply a function where we implement our program, and app_run will call it after performing initialization.
Our custom app_proc is defined on line 8. In this example, it is very basic. It defines a "canvas", just a memory area we will use to draw onto. In this example, it is small enough to fit on the stack. We clear it all to a pale gray color, and then we set the app to run in windowed mode (the default is fullscreen). Then we enter the main loop of our program, looping until app_yield tells us that the user have closed the window. app_yield gives the internal app.h code and the operating system a chance to do some housekeeping and updates. Lines 18-21 picks a random position and a random color, and plots a singel pixel there. This is where we would normally be rendering sprites and other things in a real app. Line 24 passes our canvas to app.h, to display it to the user. And then we repeat the loop.
As the example doesn't have any dependencies (other than the C runtime library), we can build it by simply sticking the above code into a main.cpp file, and typing from a visual studio command prompt:
without needing any build configurations or build scripts.
There's a bunch of other functions in app.h too, but they should be fairly well explained in the documentation section of the header file, so I will not go into more detail here - but if anyone have any questions or comments, I will be happy to elaborate.
As I have mentioned above, one of my goals with this rewrite have been to make sure each core part of the engine can be used as a standalone library. First out, is a library I call app.h, which can be found here:
https://github.com/mattiasgustavsson/libs/blob/master/app.h
app.h is the basic "platform layer" for my engine. It handles creating a window, reading input, displaying pixels and playing sound samples. I guess you could say it is doing the things one would normally use something like SDL for, but only the subset which is actually platform dependent - all the "nice to have" things like loading and rendering bitmaps, drawing fonts, loading soundfiles etc, are intentionally left out.
This is probably worth talking a bit about, as it is somewhat central to the way I approach development. When writing a library, or a module for a program, there are different things that can be prioritized. I think it is essential to have the priorities clear before you start writing the code. This is not to say that you need to know what code to write - I am a big believer in explorative programming. But you need to know the priorities. I think it is too common that programmers default to prioritizing ease-of-use above everything else, and for all the code they write. But there are other factors that can be more important, depending on circumstances. For example, sometimes performance is more important than ease-of-use. Sometimes control is at the top, maximizing the level of control the user have over the libraries functionality. Or, as is the case with the platform layer, ease of porting can be the top priority.
As can be seen in app.h, the API is very minimalistic. At various points of development, it used to have a lot more functionality, many more nice to haves. But all that would give me here, is more code to port to other platforms.
A much better approach, is to keep this library very small and minimalistic, and then build other libraries that can provide ease-of-use, which can run on top of this. The challenge is to minimize dependencies between libs. I don't want the higher level libraries to have direct dependencies on app.h - instead, I want to structure it so that when I write a program, I can pull in app.h and some other libraries and write some simple "glue code" to connect them. Keeping libraries independent like that makes it much easier to re-use them for different purposes. If I write, for example, a font renderer, I would want to be able to plug it into either app.h or some other system, like maybe SDL, depending on what I am making.
We will see later how to create other libraries on top of app.h. But even though I have libs to make it easier to use, I often use app.h directly for many smaller projects, where what I'm doing is more specialized, and where I don't particularly need the ease of use of the higher level libs. In return, I get a smaller code base and less overhead.
The library itself is a single-file header-only library like Sean Barrets stb libraries. A basic example of its use could look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | #define APP_IMPLEMENTATION #define APP_WINDOWS #include "app.h" #include <stdlib.h> // for rand and __argc/__argv #include <string.h> // for memset int app_proc( app_t* app, void* ) { APP_U32 canvas[ 320 * 200 ]; // a place for us to draw stuff memset( canvas, 0xC0, sizeof( canvas ) ); // clear to grey app_screenmode( app, APP_SCREENMODE_WINDOW ); // keep running until the user close the window while( app_yield( app ) != APP_STATE_EXIT_REQUESTED ) { // plot a random pixel on the canvas int x = rand() % 320; int y = rand() % 200; int color = rand() | ( rand() << 16 ); canvas[ x + y * 320 ] = color; // display the canvas app_present( app, canvas, 320, 200, 0xffffff, 0x000000 ); } return 0; } int main( int argc, char** argv ) { (void) argc, argv; return app_run( app_proc, NULL, NULL, NULL, NULL ); } // pass-through so the program will build with either /SUBSYSTEM:WINDOWS or /SUBSYSTEN:CONSOLE extern "C" int __stdcall WinMain( struct HINSTANCE__*, struct HINSTANCE__*, char*, int ) { return main( __argc, __argv ); } |
The first line is the typical header-only define to get the implementation. The next line, defining APP_WINDOWS, selects the platform we are running on. At the moment, only windows is supported, but I'll get around to porting it to Linux and Mac too at some point.
The action starts at line 32, in the entry point of the program, standard "main" function. We simply call app_run, and pass in a function pointer to our custom "app_proc". The app_proc is simply a function where we implement our program, and app_run will call it after performing initialization.
Our custom app_proc is defined on line 8. In this example, it is very basic. It defines a "canvas", just a memory area we will use to draw onto. In this example, it is small enough to fit on the stack. We clear it all to a pale gray color, and then we set the app to run in windowed mode (the default is fullscreen). Then we enter the main loop of our program, looping until app_yield tells us that the user have closed the window. app_yield gives the internal app.h code and the operating system a chance to do some housekeeping and updates. Lines 18-21 picks a random position and a random color, and plots a singel pixel there. This is where we would normally be rendering sprites and other things in a real app. Line 24 passes our canvas to app.h, to display it to the user. And then we repeat the loop.
As the example doesn't have any dependencies (other than the C runtime library), we can build it by simply sticking the above code into a main.cpp file, and typing from a visual studio command prompt:
1 | cl main.cpp |
without needing any build configurations or build scripts.
There's a bunch of other functions in app.h too, but they should be fairly well explained in the documentation section of the header file, so I will not go into more detail here - but if anyone have any questions or comments, I will be happy to elaborate.
Ok, so it's time for another module. Again, a stand-alone single header library for C/C++, this time for doing multi-threading in a cross-platform way. Now, if you're using C++11 (which I am not), there's <thread> from STL, and if you are using C11 (which I am not), there's <thread.h>. Windows of course has its native functions, and on linux/mac there's <pthread.h>, but I wanted something reuseable which wrapped those native functions in a consistent way. So here is my thread.h library:
https://github.com/mattiasgustavsson/libs/blob/master/thread.h
I wanted the lib to be fairly complete, as in provide all the functions I actually need, but I also didn't want a lot of bloat, so tried to keep it fairly lean. I guess the one thing that stands out as being perhaps a bit redundant, is the inclusion of a single-producer/single-consumer lock free queue, but seeing as it adds so little code and it being such a useful construct, I decided to include it anyway.
Now, I am actually not that fond of threading to begin with. If at all possible, I prefer solving problems without threading - I'd rather write my algorithms such that it is possible to specify the number of iterations it should run, and make it reentrant so it can continue where it left off, rather than offloading it to a different thread. On the other hand, when there is a need to utilize multiple cores for performance reason, that's when the threads come out.
Anyway, here is a basic example of using some of the functions from thread.h:
This sample just creates a second thread, does some printouts and then exits. The lib support more function of course. It lets you create threads, mutexes and signals (used when a thread needs to wait for an event happening on another thread). It has wrappers for performing atomic operations, both on integers and pointer variables. It has support for waiting on a high precision timer (sub-millisecond precison), and for working with thread-local storage (TLS). And also the aforementioned lock-free queue. I am generally very careful about using lock-free techniques, as it is so tricky to get them error free, so this queue is most likely the only lock-free technique I'll be using. Just hoping I got it right :) (Not seen any problems with it yet, but you never know).
https://github.com/mattiasgustavsson/libs/blob/master/thread.h
I wanted the lib to be fairly complete, as in provide all the functions I actually need, but I also didn't want a lot of bloat, so tried to keep it fairly lean. I guess the one thing that stands out as being perhaps a bit redundant, is the inclusion of a single-producer/single-consumer lock free queue, but seeing as it adds so little code and it being such a useful construct, I decided to include it anyway.
Now, I am actually not that fond of threading to begin with. If at all possible, I prefer solving problems without threading - I'd rather write my algorithms such that it is possible to specify the number of iterations it should run, and make it reentrant so it can continue where it left off, rather than offloading it to a different thread. On the other hand, when there is a need to utilize multiple cores for performance reason, that's when the threads come out.
Anyway, here is a basic example of using some of the functions from thread.h:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | #define THREAD_IMPLEMENTATION #include "thread.h" #include <stdio.h> // for printf int thread_proc( void* user_data) { thread_timer_t timer; thread_timer_init( &timer ); int count = 0; thread_atomic_int_t* exit_flag = (thread_atomic_int_t*) user_data; while( thread_atomic_int_load( exit_flag ) == 0 ) { printf( "Thread... " ); thread_timer_wait( &timer, 1000000000 ); // sleep for a second ++count; } thread_timer_term( &timer ); printf( "Done\n" ); return count; } int main( int argc, char** argv ) { (void) argc, argv; thread_atomic_int_t exit_flag; thread_atomic_int_store( &exit_flag, 0 ); thread_ptr_t thread = thread_create( thread_proc, &exit_flag, "Example thread", THREAD_STACK_SIZE_DEFAULT ); thread_timer_t timer; thread_timer_init( &timer ); for( int i = 0; i < 5; ++i ) { printf( "Main... " ); thread_timer_wait( &timer, 2000000000 ); // sleep for two seconds } thread_timer_term( &timer ); thread_atomic_int_store( &exit_flag, 1 ); // signal thread to exit int retval = thread_join( thread ); printf( "Count: %d\n", retval ); thread_destroy( thread ); return retval; } |
This sample just creates a second thread, does some printouts and then exits. The lib support more function of course. It lets you create threads, mutexes and signals (used when a thread needs to wait for an event happening on another thread). It has wrappers for performing atomic operations, both on integers and pointer variables. It has support for waiting on a high precision timer (sub-millisecond precison), and for working with thread-local storage (TLS). And also the aforementioned lock-free queue. I am generally very careful about using lock-free techniques, as it is so tricky to get them error free, so this queue is most likely the only lock-free technique I'll be using. Just hoping I got it right :) (Not seen any problems with it yet, but you never know).
You can set stack size for posix thread's with pthread_attr_setstacksize function.
There are few more threading primitves you are missing
- Windows events. I find them very useful in many situations. On Linux this can be done with eventfd. No idea about macOS. Cool stuff about events and Linux eventfd is that you can wait on multiple handles at the same time.
- Semaphores. Also useful.
There are few more threading primitves you are missing
- Windows events. I find them very useful in many situations. On Linux this can be done with eventfd. No idea about macOS. Cool stuff about events and Linux eventfd is that you can wait on multiple handles at the same time.
- Semaphores. Also useful.
Ah, yeah, I should definitely do the stack size for pthread as well. Thanks!
I realize that it might not have all the threading primitives - I've just added the ones I actually use for my current projects. I don't mind if others expand on what I have hear, but I would be concerned about bloat, so there's probably some things I would choose not to put into this lib (for example, more complex sync queues and similar).
I guess I'd use my "signal" concept instead of events. Though you can't wait for multiple on that one.
I realize that it might not have all the threading primitives - I've just added the ones I actually use for my current projects. I don't mind if others expand on what I have hear, but I would be concerned about bloat, so there's probably some things I would choose not to put into this lib (for example, more complex sync queues and similar).
I guess I'd use my "signal" concept instead of events. Though you can't wait for multiple on that one.
Next up, managing strings.
Now, strings is something I have mixed feelings about. I've worked on enough game projects whith real bad misuse of strings to be rather wary of them. I remember one game I worked on, this is some years ago, so the min spec was a windows machine with 256 MB of memory. When I joined the project, it was using almost 1 GB of memory, and I was tasked to get it down. Turned out it was using over 380 MB of strings. Or rather, multiple copies of the same string. The data driven gameplay systems used strings to identify, well, pretty much everything, and it used std::string from the STL to store them everywhere. I replaced it by a simple string pool (also known as string interning), storing each unique string only once, and having all other systems just store handles to the real strings. This reduced string memory use from 380 MB to some 100 KB. That's the first time I used a string pool, but not the last.
Generally speaking, if you are making a big game with a large team, I would insist that you don't use strings for *anything* other than text which is displayed to the user. Just pre-process your data and use hashes or integer ID's for everything. More efficient and less error prone.
When I'm making small games on my own though, I find it really convenient to use strings as IDs directly, not baking things out to hashes in pre-processing steps. But, I don't like the performance hit you get from using, for example, std::string, where every string compare means looping over the whole string (and also cache misses following the pointer to the string data).
So this weeks single header library is a highly efficient, generic string pool/string interning system written in C, useable from C and C++.
https://github.com/mattiasgustavsson/libs/blob/master/strpool.h
Here's an example of how to use it:
There's a lot of details in the documentation describing each function in the API.
The actual implementation is something I have worked on on and off for a long time. The current version is an evolution of code I started many years ago, and I dare say it is very efficient at dealing with strings. The API can feel a little primitive, especially if using the support for reference counting. What I usually do though, when I use the string pool in a sizeable project, is to use a C++ wrapper for it, to automate some of the more laborious or error prone aspects. In a later posting, I will share my generic wrapper, as well as some utility functions I find very handy - to the point where I never even consider using std::string again, for anything.
Now, strings is something I have mixed feelings about. I've worked on enough game projects whith real bad misuse of strings to be rather wary of them. I remember one game I worked on, this is some years ago, so the min spec was a windows machine with 256 MB of memory. When I joined the project, it was using almost 1 GB of memory, and I was tasked to get it down. Turned out it was using over 380 MB of strings. Or rather, multiple copies of the same string. The data driven gameplay systems used strings to identify, well, pretty much everything, and it used std::string from the STL to store them everywhere. I replaced it by a simple string pool (also known as string interning), storing each unique string only once, and having all other systems just store handles to the real strings. This reduced string memory use from 380 MB to some 100 KB. That's the first time I used a string pool, but not the last.
Generally speaking, if you are making a big game with a large team, I would insist that you don't use strings for *anything* other than text which is displayed to the user. Just pre-process your data and use hashes or integer ID's for everything. More efficient and less error prone.
When I'm making small games on my own though, I find it really convenient to use strings as IDs directly, not baking things out to hashes in pre-processing steps. But, I don't like the performance hit you get from using, for example, std::string, where every string compare means looping over the whole string (and also cache misses following the pointer to the string data).
So this weeks single header library is a highly efficient, generic string pool/string interning system written in C, useable from C and C++.
https://github.com/mattiasgustavsson/libs/blob/master/strpool.h
Here's an example of how to use it:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | #define STRPOOL_IMPLEMENTATION #include "strpool.h" #include <stdio.h> // for printf #include <string.h> // for strlen int main( int argc, char** argv ) { (void) argc, argv; strpool_config_t conf = strpool_default_config; //conf.ignore_case = true; strpool_t pool; strpool_init( &pool, &conf ); STRPOOL_U64 str_a = strpool_inject( &pool, "This is a test string", (int) strlen( "This is a test string" ) ); STRPOOL_U64 str_b = strpool_inject( &pool, "THIS IS A TEST STRING", (int) strlen( "THIS IS A TEST STRING" ) ); printf( "%s\n", strpool_cstr( &pool, str_a ) ); printf( "%s\n", strpool_cstr( &pool, str_b ) ); printf( "%s\n", str_a == str_b ? "Strings are the same" : "Strings are different" ); strpool_term( &pool ); return 0; } |
There's a lot of details in the documentation describing each function in the API.
The actual implementation is something I have worked on on and off for a long time. The current version is an evolution of code I started many years ago, and I dare say it is very efficient at dealing with strings. The API can feel a little primitive, especially if using the support for reference counting. What I usually do though, when I use the string pool in a sizeable project, is to use a C++ wrapper for it, to automate some of the more laborious or error prone aspects. In a later posting, I will share my generic wrapper, as well as some utility functions I find very handy - to the point where I never even consider using std::string again, for anything.
thank you.
i started a blank VS 2015 Community project, added app.h and copied the sample from the documentation into a main.cpp. And it just worked. made me so happy i had to make my first comment on HMN : )
wishing you all the best.
i started a blank VS 2015 Community project, added app.h and copied the sample from the documentation into a main.cpp. And it just worked. made me so happy i had to make my first comment on HMN : )
wishing you all the best.
That's awesome :) I have worked hard to make it as easy as possible to get working. If you get stuck on anything, just let me know, happy to answer questions about it.
So, for the fourth weekend in a row (I am on a roll) I give you another stand-alone library. I have quite a few of these to go through before we get to "pixie.hpp" which is the single-header wrapper which ties all these libraries together into a game engine lib. But that one will be quite boring by then, as it won't do much on its own - all the magic happens in libs.
This week, a simpler one - reading and writing .ini files.
https://github.com/mattiasgustavsson/libs/blob/master/ini.h
This one is quite straightforward. You can pass it a string, and it will parse its .ini content. Like this:
Or you can build up a data structure, and save it out to a string buffer, like this:
Note that the library doesn't do any file operations - it just works on memory passed to it. I think this is much better, as it allows the caller to read the data using any method. I like keeping libs small and focused.
So why .ini files you might ask? Obviously its basically just personal preference, but allow me to reminisce for a bit. I first got exposed to .ini files in 1993, when MS-DOS 6 came out, and I'd only just gotten my first PC (a 486DX2/66). Before that, I'd been on Atari ST, and text-based config files just weren't a thing back then. We just used binary data for everything.
There was an appeal to using text-based config files though, and .ini files set a convenient standard. A couple of years later, when I made my first Windows95 game using DirectX3, it was just natural to use .ini files to configure my data. There was built-in support for it in the windows API after all. I think I went a bit overboard though, as I used .ini files for everything, even level files. It worked well enough though.
Moving with times, I soon came to use XML. We all did for a while. XML for everything. It was horribly overcomplicated but we didn't realize. Shudder. Eventually came json and yaml, and they were nicer than XML, but still... Thinking back on it, none of them has been as simple and straightforward as the .ini files of old. Yes, that's my personal opinion, but still :)
Sure, they are not as flexible, and can't store hierarchical structures as well. But for the things I do, which is small games, I don't event WANT to store such things in text format. I just make an efficient custom binary format and leave it at that.
For some things though, I want a human editable format, for both me and my users to be able to change configuration settings - and for this, I don't think anything beats .ini files - they are just so intuitive, and there are few rules to them.
So this lib is my custom implementation for parsing .ini files, since I don't want to rely on built in functionality of the operating system, like I did back in -96.
The library is fairly efficient in how it manages memory. It makes dynamic allocations, but try to keep the number of allocations down. This is done by keeping all sections in one block, and all properties, regardless of which section they belong to, in another block, with data pointing out which section each property belongs to. The name and value strings stored for properties, and the name strings for sections, are stored within these blocks if the strings are short (less then 32 chars for names, less than 64 for values), and only allocated from the heap if they are longer.
I still wouldn't recommend using ini files for large amounts of data though. But for small configurations, they are perfect :)
This week, a simpler one - reading and writing .ini files.
https://github.com/mattiasgustavsson/libs/blob/master/ini.h
This one is quite straightforward. You can pass it a string, and it will parse its .ini content. Like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | #define INI_IMPLEMENTATION #include "ini.h" #include <stdio.h> #include <stdlib.h> int main() { FILE* fp = fopen( "test.ini", "r" ); fseek( fp, 0, SEEK_END ); int size = ftell( fp ); fseek( fp, 0, SEEK_SET ); char* data = (char*) malloc( size + 1 ); fread( data, 1, size, fp ); data[ size ] = '\0'; fclose( fp ); ini_t* ini = ini_load( data ); free( data ); int second_index = ini_find_property( ini, INI_GLOBAL_SECTION, "SecondSetting" ); char const* second = ini_property_value( ini, INI_GLOBAL_SECTION, second_index ); printf( "%s=%s\n", "SecondSetting", second ); int section = ini_find_section( ini, "MySection" ); int third_index = ini_find_property( ini, section, "ThirdSetting" ); char const* third = ini_property_value( ini, section, third_index ); printf( "%s=%s\n", "ThirdSetting", third ); ini_destroy( ini ); return 0; } |
Or you can build up a data structure, and save it out to a string buffer, like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | #define INI_IMPLEMENTATION #include "ini.h" #include <stdio.h> #include <stdlib.h> int main() { ini_t* ini = ini_create(); ini_property_add( ini, INI_GLOBAL_SECTION, "FirstSetting", "Test" ); ini_property_add( ini, INI_GLOBAL_SECTION, "SecondSetting", "2" ); int section = ini_section_add( ini, "MySection" ); ini_property_add( ini, section, "ThirdSetting", "Three" ); int size = ini_save( ini, NULL, 0 ); // Find the size needed char* data = (char*) malloc( size ); size = ini_save( ini, data, size ); // Actually save the file ini_destroy( ini ); FILE* fp = fopen( "test.ini", "w" ); fwrite( data, 1, size, fp ); fclose( fp ); free( data ); return 0; } |
Note that the library doesn't do any file operations - it just works on memory passed to it. I think this is much better, as it allows the caller to read the data using any method. I like keeping libs small and focused.
So why .ini files you might ask? Obviously its basically just personal preference, but allow me to reminisce for a bit. I first got exposed to .ini files in 1993, when MS-DOS 6 came out, and I'd only just gotten my first PC (a 486DX2/66). Before that, I'd been on Atari ST, and text-based config files just weren't a thing back then. We just used binary data for everything.
There was an appeal to using text-based config files though, and .ini files set a convenient standard. A couple of years later, when I made my first Windows95 game using DirectX3, it was just natural to use .ini files to configure my data. There was built-in support for it in the windows API after all. I think I went a bit overboard though, as I used .ini files for everything, even level files. It worked well enough though.
Moving with times, I soon came to use XML. We all did for a while. XML for everything. It was horribly overcomplicated but we didn't realize. Shudder. Eventually came json and yaml, and they were nicer than XML, but still... Thinking back on it, none of them has been as simple and straightforward as the .ini files of old. Yes, that's my personal opinion, but still :)
Sure, they are not as flexible, and can't store hierarchical structures as well. But for the things I do, which is small games, I don't event WANT to store such things in text format. I just make an efficient custom binary format and leave it at that.
For some things though, I want a human editable format, for both me and my users to be able to change configuration settings - and for this, I don't think anything beats .ini files - they are just so intuitive, and there are few rules to them.
So this lib is my custom implementation for parsing .ini files, since I don't want to rely on built in functionality of the operating system, like I did back in -96.
The library is fairly efficient in how it manages memory. It makes dynamic allocations, but try to keep the number of allocations down. This is done by keeping all sections in one block, and all properties, regardless of which section they belong to, in another block, with data pointing out which section each property belongs to. The name and value strings stored for properties, and the name strings for sections, are stored within these blocks if the strings are short (less then 32 chars for names, less than 64 for values), and only allocated from the heap if they are longer.
I still wouldn't recommend using ini files for large amounts of data though. But for small configurations, they are perfect :)
I only had one strpool.h in my single-file-library list, so I added the others you've posted about here to https://github.com/nothings/single_file_libs
Some of this looks pretty great. If it weren't for the brace style I might actually use them! :)
Some of this looks pretty great. If it weren't for the brace style I might actually use them! :)
nothings
I only had one strpool.h in my single-file-library list, so I added the others you've posted about here to https://github.com/nothings/single_file_libs
Some of this looks pretty great. If it weren't for the brace style I might actually use them! :)
The single-file-library king himself endorses your libs. Good stuff.