Register
handmade.network » Wiki » How to write better (game) libraries

Introduction

The aim of this article is to provide a series of good general advice and considerations on how to design and write libraries, particularly if portability, ease of use and performance are of concern.

The word "game" is in parentheses since most of the advice also applies to non-game libraries.

Bear in mind that there is no "true way" to write a library and different people have different opinions, however, there do exist general advice and considerations that can be examined based on existing work.

The advice and considerations presented in this article are not meant for all kinds of libraries, for example, for libraries such as language-specific containers or wrappers around os functionality, some of the advice may not apply. Remember that when designing a library you should understand your requirements and do what you find to be the best approach, this article acts simply as a non-exhaustive list of good considerations and advice.

This article aims to present advice based on existing libraries that are considered of quality in the community so that developers can better understand some of the considerations involved in designing a library. The list does not aim to be exhaustive, but hopefully, it can be updated to include more advice and considerations over time and serves as a starting point and educational resource to anyone interested in library design.

The core principles of this advice are:

  1. Maximize portability.

  2. Be easy to build.

  3. Be easy to integrate.

  4. Be usable in as many scenarios as possible.

Consider writing the library in C and add wrappers to other languages later.

C is the lingua franca of programming. There are many advantages to writing your library in C from the start:

  1. Every language out there has a way to call into C, so if you write your library in C everyone will be able to use it, in any language, and people can write wrappers for it easily. If you write a library in Python, chances are most people won’t be able to use it if they don’t also use Python. If you write a library in C however, someone who really likes it can make bindings for it in Python. C truly brings all of us together.

  2. C code is usually fast. There is a saying that if your code is slower than C, someone will rewrite it in C. Performance is especially important for game developers and people who use low-level languages in general.

  3. C is the most portable language in the world, if your library is written in C it means it can be used on any OS, console or mobile device and even on the web.

You might also consider writing the library in C++, be mindful however of the following drawbacks:

  1. It is easier in general for a C++ user to use a C library than it is for a C user to use a C++ library.

  2. C++ is not as easy to write wrappers for in other languages.

  3. If you use C++, unless you limit which C++ features you use (to the point where you are pretty much left with C) a lot of people won’t be able to use your library. Not everyone uses RAII, some people disable exceptions and RTTI, and not everyone is willing to use a library with smart pointers, template metaprogramming, STL allocators, virtual dispatch, etc.

Take into account the following aspects when writing your C library:

  1. Compiler extensions make your code less portable, try using the subset of C99 which compiles both in C and C++ mode on GCC, Clang, and MSVC. This is important since some compilers (such as MSVC) have problems with supporting C.

  2. Try not to use the hosted (OS-dependent) parts of the C standard library, unless you really have to, since they might work differently on other platforms (or might not work at all). If you need to use OS-dependent functions but want to maximize portability, request function pointers from the client.

  3. Prefix your names to avoid name collisions (more on this later).

  4. For maximum portability consider using the built-in C types (char, short, int, long). This is because not all compilers and platforms have support for stdint.h (eg: old versions of MSVC). Try checking for the availability of stdint.h and use that if available. As an example consider looking at how the library stb_image handles the use of stdint.h.

  5. Consider using header guards instead of #pragma once. Header guards are standard and also allow the user of the library to check if a library was included.

  6. Undef macros that should not be exposed to the user at the end (eg: my_min/max macros), do this even in C files.

  7. Make sure your library can compile as one compilation unit since a lot of developers choose to do single-compilation-unit builds (also known as unity builds).

If you want to use another language for the implementation, consider keeping the interface in C.

If you want to write a library using another language, such as Odin, Rust, Zig, C++, etc, consider keeping the interface in C so that people can easily use it and wrap it for other languages.

Bear in mind that this has several disadvantages:

  1. People might have a harder time integrating your library into their projects as source.

  2. You might be adding the standard library of your language of choice as an extra dependency that the user of your library now needs to consider. This can also have other implications, for example, some standard libraries use the general heap allocator, which violates the principle of giving the user of the library full control over memory allocations (more on this later). Also interfacing with certain aspects of a standard library from another language via a C API could be awkward.

  3. You might need to provide precompiled binaries or ways for your users to build your library for their target platform. This might not be an issue with C++ but it can be with other languages (eg: Odin, Zig, Rust).

  4. If people don't use the same language for their projects their debugging experience might suffer when using your library.

Consider providing an optional C++ wrapper.

Besides C users, C++ users are the ones most likely to use your C library. To that extent, you might want to consider making their experience better by providing a C++ wrapper.

Take into consideration that this is an extra piece of code you would have to maintain but that can help the adoption of your library by C++ developers.

Basic things you can do when writing your C++ wrapper:

  1. Use .hpp and .cpp for your C++ wrapper to distinguish between the C and C++ code. Consider putting the C++ wrapper in another folder or repo and mention it in the readme.

  2. Try not to include the header from the C version in the hpp file. Instead, rewrite the declarations in the hpp in a more C++ style and in the cpp file include the C header and provide the definitions for all the wrapper functions. This is to prevent the names from C and macros to spill into C++ code.

  3. Expose constants to the user using constexpr variables. This is especially easy since C structs are constexpr by default. Make sure constants from C that use macros are not present in the C++ wrapper.

  4. Use namespaces and wrap all the functions like so namespace mylib { foo bar() { return mylib_bar(); }.

  5. Try not to use exceptions and RTTI. Especially in game development, not all people use them and some just disable them.

  6. Consider using default parameter values over function wrappers. C lacks default values for parameters so usually, people write a function with a lot of parameters and then create several wrappers for it that calls the original function with different default parameter values. C++ has default parameter values so consider removing the extra wrapper functions if you have any.

  7. Try not to use STL containers or smart pointers since your wrapper should just simply wrap the functions from C and also adding those that bring extra problems that you need to consider. (More on this later)

  8. If you decide to offer RAII wrappers for parts of your library, still provide wrappers for the non-RAII structs and functions. This is important because not all developers use RAII and if you don’t expose the non-RAII versions of structs and functions in your C++ layer they won’t be able to use it, at least not fully.

Try not to make the use of a build system mandatory.

There are many build systems out there and chances are people won’t use the same one you do. If your library simply presents the source files, the header files and a list of the dependencies (if you have any), this will help developers integrate your library into their projects.

If there are compiler flags needed to compile your library then you can mention them in the readme, though preferably there won’t be any.

It is good to include optional build files, but mention in the readme that they are optional and try to keep the simple structure that lets people integrate the library easily with their preferred build system.

If you do use a build system such as CMake or Make, try to use them in a standard way so that developers who choose to use them can get the most out of them.

Ensure that people can easily compile the library from source.

Distribute your libraries such that people can simply include the source files in their project and be done with it.

This has many advantages such as:

  1. Letting people easily reason about your library.

  2. Easy to include in cross-platform projects, no craziness regarding libc versions and platform stuff.

  3. It allows people to step through the code in a debugger.

Minimize dependencies.

If your library has dependencies, try to list them in a file (such as the readme) so that users can easily reason about them. Also, try to only use dependencies if absolutely necessary. Don’t get a dependency if you only use 1 - 3 functions from it since that adds bloat. Instead, consider writing your own version or extracting those functions you need from another library into your own. For example, if you need some light text manipulation functions then consider writing your own instead of adding a dependency, but if you need to parse a JSON file, a well-tested library might be worth using instead of trying to implement your own.

Try not to fall into the trap of using a library for every single small thing (eg: see is-odd and is-even from NPM).

The more dependencies you have the harder it will be for the user to decide whether they should use your library because they now have to consider all of your dependencies as well.

Try not to allocate memory for the user.

Many developers, especially in game dev and low-level software development, have custom memory allocation strategies and do not use the general heap allocator (malloc/free). If a library allocates memory using the general-purpose heap allocator then certain developers might not be able to use it.

To make the experience better for them, consider asking for buffers or allocators from the user instead of calling malloc/free yourself and use stack-allocated buffers (up to one or more kb) for small temporary operations (eg: some string manipulation).

Also, if you decide to write your library in C++, using STL containers, without a way for developers to specify the allocator, this would also make it unusable for them, which is another reason to prefer C99 and write the C++ wrapper afterward.

If your library needs to do a lot of extra work and has a lot of tiny allocations that can occur (such as for temporary allocations when doing string manipulation), consider asking the user for a buffer or an allocator for temporary memory.

In C++ you can use default parameters for functions that ask for an allocator to use the heap allocator by default. You can also overload functions to have a version that allocates the memory for the user.

Be const correct.

Some developers use const, some don’t. If you don’t use const for the function parameters in your functions however, this will only negatively affect developers who use choose to use const. So use const for pointers to data that you intend to only read from. Being const correct also helps with documenting code, it even helps developers who don’t use const.

Always ask for the size of buffers (even for const char*)

Since we mentioned custom allocators and asking for buffers instead of allocating memory using the heap allocator, remember to always ask the user for the size of the provided buffer. Even if the buffer you receive is expected to have a sentinel value (such as a null terminator) still ask for the size of the buffer. The reason is, in game development and other high-performance software, we often pack buffers together in memory for efficiency reasons, so going out of the bounds of a buffer might not result in a crash from a memory access violation and this is a very hard to catch bug. Also, not all users use null-terminated strings all the time (for a variety of reasons). If you must use a null-terminated string, such as when interfacing with an OS function, consider informing the user about this and still maybe ask for the buffer size.

When dealing with strings also bear in mind that buffer size and string size are different, since the string size doesn’t take into consideration the null terminator, so specify in comments and documentation if the user should provide the buffer size or the string size when asking for a const char*.

Additionally, you can provide convenience functions that ask for a C Null Terminated String and invoke strlen to get the size and pass it to the function they wrap.

Try not to handle resources for user.

Write your library such that a user can simply pass everything that it needs to it and then it gives the user back what they wanted. If your library needs something, consider asking the user for it instead of trying to get that thing on your own.

Try not to handle memory or other resources in a way that is invisible to the user since they might encounter problems if the way you manage that resource is incompatible with that they are doing.

Consider offering optional versions of your functions that handle resources or allocate resources on behalf of the user. The user might want to use those (if it’s possible in their case) or at the very least they can use those to test out the library.

Avoid using os-dependent functions (such as fopen) for maximum portability.

This might sound weird but sadly fopen and many other os-dependent functions don't work as intended on all platforms that a game dev might want to support.

For example, on Android, you need to use AAssetManager for assets and fopen for internal storage. Because of this, in order to use libraries that need fopen on Android, some devs do stuff like #define fopen which is really hacky and error-prone. Also depending on the project, IO must be handled in different ways (eg: memory-mapped files, async IO).

Instead, you could do the following things:

  1. Ask for buffers to the file data and let the client of the library do the IO.

  2. Ask for function pointers for IO operations that the client of the library needs to provide.

  3. Offer optional variants of those functions that use fopen, most games are tested and developed on Windows or Linux where such functions can be used to integrate the library into the game quickly for testing purposes.

In general, if you are unsure about the availability and support of a os-dependent function in libc and don’t want to have to manually provide support for every single platform that users might need (especially if you only need a few os-dependent functions), consider asking the user for function pointers to functions that achieve the goal of that os-dependent function. This maximizes your portability and it is simple for your users to do.

Don’t load OpenGL in the library if you depend on it.

If your library depends on OpenGL then don’t include any OpenGL headers and don’t try to load OpenGL yourself. Sadly OpenGL needs to be loaded very differently on each platform and it’s hard to load it without essentially enforcing a platform layer and maintaining it for all platforms. Not to mention on some platforms there are multiple, mutually exclusive ways to load OpenGL and depending on the scenario a game dev might use one over the other.

If your library handles OpenGL loading by itself then people can’t use it, not without having to fork it to remove your custom OpenGL loading code. Depending on another library which also does this is also bad.

In your code, don’t include any OpenGL headers. Instead, specify in the readme which version of OpenGL you use and tell users which .c files need OpenGL. What they can then do is wrap those files like such:

1
2
3
// library_file_that_uses_opengl.c
#include <my_opengl> // Include a header with OpenGL definitions that works for my game
#include <library/file_that_uses_opengl.c>

This way users can compile your library which uses OpenGL and have it work with how they load OpenGL in their game.

For DirectX and Metal, there are standard headers and you can check if the user defined a flag like MYLIB_USE_DIRECTX or MYLIB_USE_METAL and include those headers based on that. For Vulkan, I think you need to do the same thing as with OpenGL.

For an example of this pattern take a look at sokol_gfx

For higher portability when writing rendering code, consider using the subset of OpenGL from OpenGL ES 3, unless you are willing to provide multiple versions of your graphics code.

If possible, offer support for multiple graphics backends like DirectX and Metal. If you can only focus on adding one extra backend prefer Metal since OpenGL is deprecated on iOS and macOS. When writing the code for multiple different graphics backends or API versions, consider splitting the rendering code into different files depending on the backend and API version (eg: Project files: my_lib.h, my_lib_opengles2.c, my_lib_opengl4.c, my_lib_metal.c).

Provide examples.

Examples are a great way for people to get to know your library.

  1. Providing a folder with easy to compile 1-2 file example source code is amazing.

  2. Providing examples in the comments is awesome.

  3. Providing examples in the readme is great.

  4. Providing a Github wiki (though not necessary for most projects) is god-like.

If your library is easy to understand and easy to compile that increases the chances of anyone using it exponentially.

Another advantage of a suite of examples is that it can serve a double purpose as a testing suite. If you have lots of examples you can use them to validate that your library still works correctly. This is especially important for libraries that do rendering since you can’t normally unit test that.

Use namespace prefixes.

Prefix any exported symbol like such mylib_ to avoid naming collisions. This includes macros, enum constants, function/struct names. Do this in the .c file as well since some people do single-compilation-unit builds. Consider using a different naming convention for internal functions that should not be exposed to the user (eg: mylib_impl_ or mylib__), this helps people who do single-compilation-unit builds.

Avoid global variables if possible.

There are moments when global variables are appropriate, such as for static data that must be available to the library. Global variables are not always the worst thing, and most people can tolerate them depending on their use (sometimes they even make the API easier to use).

But try to keep them to a minimum in general as they can have implications:

  1. What if the user tries to use your library on multiple threads?

  2. Many people do hot reloading by compiling the game code into a DLL that gets refreshed on recompile while the game executable still runs. When they do this all global variables are reset. This means they can’t use your library in game code if they do hot reloading.

Consider what error handling strategy you will use.

There are many ways to handle errors in C and there isn't just one standard way, take careful consideration as to what error handling mechanism you will use. It is also important to distinguish between real errors and precondition/invariant/postcondition violations.

Precondition, invariant and postcondition violations are not errors.

These are usually bugs, and it is best to handle them using assertions. Use a macro like #define MYLIB_ASSERT(condition, msg) assert(condition) which is set by default to the libc assert but that the user of the library can modify to better fit their needs or to disable assertions for performance.

Don’t offer error handling unless there is actually an error to recover from.

Debuggers can catch assertion failures and the developer can see the stack trace.

Additionally, messages and comments regarding what triggered the assert are great, do those.

Consider error codes or result structs for errors that must be handled at runtime by the user

If you do something that can fail and that needs to be handled by the client of the library (eg: you try to send a package over the internet but it fails) then simply return error codes (preferably enum constants) to the user. If you have additional information that is important to the user for the goal of handling the error then return a struct with the error code and extra data, but this is usually not the case. This is often the most popular and simple choice for error handling in C.

Example:

1
2
ParsePNGFileResult result = ParsePNGFile(png_file_data);
if (result.error) { /* handle error */ }

Consider keeping error information in handles/context structs

For a series of functions that use a handle or a context struct, you might want to consider including the error in the handle and return early instead of asserting if an error is set in the handle/context struct. This gives users maximum flexibility because they can either check for an error after every single function call (like with returning error codes) or they can execute a full code path and check for errors at the end.

Example:

1
2
3
4
5
6
FileHandle file = OpenFile(file); //The file handle could be invalid after this, that information is stored in the file handle
data = GetDataFromFile(file); //If the file handle is invalid the function just returns NULL
DoStuffWithData(data); // Note that if data is null the function just returns and does nothing
CloseFile(file);

if (CheckError(file)) { /* handle error */ }

In this example, the user had the option to either handle the error after an individual function call or at the end of the full code path. This pattern is used by many developers and by designing your library’s error handling this way you can satisfy both people who want to handle errors at the call site and people who want to handle it later on.

Here is an example of how you can refactor that code to check the error at the call site:

1
2
3
FileHandle file = OpenFile(file); //The file handle could be invalid after this, that information is stored in the file handle
if (CheckError(file)) { /* handle error */ }
...

Note that when you compare this example to the one where we returned an error code it is not that different, but in practice this makes error handling much more flexible when dealing with handles or context structs.

It is important to remember however that this method only works when you have a handle or a context struct, if your function has no context or handle then consider using an error return value.

Other error handling techniques

There are many other error handling techniques available. If you find other techniques to be more appropriate for your use case consider those. The ones mentioned above are merely some of the more popular options. Other options include errno, error handlers, exceptions (in C++).

Consider your distribution model.

Libraries like STB popularised the idea of Header Only Libraries due to the following factors:

  1. They are easy to build.

  2. They are easy to include in a project.

  3. They are easy to share and reason about because there is only one file.

However, when deciding how to distribute your library consider the following drawbacks that a single header library could have as well:

  1. Header only libs can lead to longer compilation times because the whole file needs to be included even if the implementation is then discarded.

  2. Some people prefer the implementation and interface to be kept separate for organizational and build other purposes.

  3. It’s not much harder to do header + source than it is to do header only.

At the end of the day keeping your library header-only will probably not be a deal-breaker, but as with everything, make sure to take pros and cons into consideration.

Conclusion

Most of the advice presented in this article is based on feedback from the community and analysis of some of the most popular C libraries used by developers. Take into consideration that many people will have different opinions and this is ok. The article is meant to be a collection of advice and considerations based on the analysis of existing libraries. It is meant to be a starting point for people interested in developing libraries and the considerations that they might have to take when writing them.

You can check out libraries analyzed for this article here:

  1. Sokol libraries (graphics, platform layer, audio, net): https://github.com/floooh/sokol

  2. BGFX (graphics): https://github.com/bkaradzic/bgfx

  3. STB libraries (asset loading, utilities): https://github.com/nothings/stb

  4. Cuteheaders (graphics, net, io, utilities, etc): https://github.com/RandyGaul/cuteheaders

  5. CGLTF (glTF loading): https://github.com/jkuhlmann/cgltf

  6. DR Libs (audio loading): https://github.com/mackron/dr_libs

  7. Mini audio (audio playback): https://github.com/dr-soft/miniaudio

  8. Par headers (graphics, utilities, etc): https://github.com/prideout/par

  9. Physac (physics): https://github.com/victorfisac/Physac

  10. Dear Imgui (ui): https://github.com/ocornut/imgui

  11. Soloud (audio): https://github.com/jarikomppa/soloud

  12. Meow Hash (non-crypto hash): https://github.com/cmuratori/meow_hash

More to be added...