A New Asset System (9/10/2018)

Ryan Fleury
Recently, I've been working on the game's asset system. I'm doing this to start allowing other developers to have more immediate power in the process of making the game; if my artist would like to add a new texture for a new static object to decorate the world more, they should be able to do that without having to go through me. Similarly, if my music artist who is working on background music is spending time in the world and finds that a new ambient music pattern would suit the environment well, they should be able to add it and see the changes immediately.

When I started approaching this problem, I soon realized that it wasn't necessarily straightforward, however; there are times that the game's compiled code needed to reference a texture directly (for example, when drawing or animating the player). The code referenced constants that were the index of the certain textures directly. This is not unreasonable; from the perspective of he who is writing the code, it is perfectly readable, and it is as fast as hard-coding the index of the texture (because the constant is resolved to the equivalent literal at compile-time).

1
2
3
4
5
6
7
8
enum {
	// ...
	TEX_player,
	// ...
	MAX_TEX
};

Texture *player_texture = texture(TEX_player);


There's a problem, though. If I want my artist to be able to add new assets at run-time, I cannot guarantee a reliability of texture indices; they very well might change at run-time. In such a case, the code would still refer to the texture by an index equal to whatever TEX_player resolves into. This would not work. Anything more complicated, however, would add run-time overhead, and perhaps a lot.

This is when that mysterious light bulb appeared over my head. When I'm working on the game, I don't care about performance to an extent that I would when preparing a shipped version. I want the game to work quickly enough to test effectively, and I don't care about much else. I already had a build mode for both a release version and a debug version; I could just do slow (but convenient) things in debug mode, but translate those things to something fast in release mode.

There are a few challenges here. Firstly, there is the actual implementation of the dynamic asset sets. I then had the idea of "asset tags", which are a way to express intent for an asset without assigning it to a texture. There had to be some way within the asset system's API to say "get me the player texture, whatever it turns out to be"; tags allow the programmer to do this. Secondly, there is a code flexibility problem. The following could work:

1
2
3
4
5
#if BUILD_RELEASE
Texture *player_texture = texture(TEX_player);
#else
Texture *player_texture = texture_from_tag_table("player");
#endif


...but that is extremely cumbersome, and assumes that "player" refers to TEX_player. In this case, that wouldn't change (I will always have the player texture map to "player"), but this might not be the case elsewhere, especially after modifications have been made by my artist (who might add new tags, textures, etc.). This is not a maintainable solution.

My thoughts took me to the realization that I had an API problem; what is an API that could resolve both into something that looks up into a set of tags at run-time in developer mode, but resolves to a constant at compile-time in release mode?

I solved this problem by introducing the following:

1
2
3
4
#define assets_get_texture_by_tag(assets, name) // Insert something here...

// This can be used like:
Texture *player_texture = assets_get_texture_by_tag(assets, player);


What is the macro assets_get_texture_by_tag defined as, then?

1
2
3
4
5
#if BUILD_RELEASE
#define assets_get_texture_by_tag(assets, name) (&(assets)->textures[TEX_TAG_ ## name])
#else
#define assets_get_texture_by_tag(assets, name) (&(assets)->textures[asset_index(assets->texture_tag_table, #name)])
#endif


In release mode, it resolves to a pointer to the texture that's at index TEX_TAG_player, but in developer mode, it performs the needed tag look-up.

TEX_TAG_player, then, is just defined as the index of the texture that "player" mapped to at compile-time! This is done with a simple metaprogram that takes:

1
2
"player" : "player.png"
# etc.


...and generates:

1
#define TEX_TAG_player TEX_player


...along with a file defining the indices for different textures (like TEX_player).

The above system produces texture-grabbing that is just as fast as it was before in release mode, but allows for run-time modifications in developer mode! As the system is implemented currently, these modifications can be which texture each tag maps to, or the texture itself.

Here's a few videos of this system in action:





As you can see, as I make modifications to the tags file, the texture being used for the game's splash screen changes at run-time (because the game is referring simply to the texture that is mapped to "splash", instead of any specific texture directly). Additionally, the game reloads textures when they have been modified.

That's the new asset system. It seems to be a great addition, and will hopefully benefit the productivity of the game's developers. I hope you enjoyed the read!

Ryan

Comments

Nice! You mentioned other game developers working on it, are there more than you programming and how much will the artist and music developer have in the game design input?
OliverMarsh
Nice! You mentioned other game developers working on it, are there more than you programming and how much will the artist and music developer have in the game design input?


Hey there, Oliver!

I am, as of now, the only programmer, though I'll probably have someone else port the platform layer in some cases.

I try to give my artist and musician relatively significant artistic freedom in how they interpret my ideas, mostly because it can often provide valuable perspective, feelings, and thoughts about what I perceive the game's world and atmosphere to be. That being said, I'll be doing the majority of level design and gameplay design, though input from the artist and musician is of course always accepted and appreciated. These tools are mostly for the purpose of empowering my artist and musician in their respective roles regarding asset development. That being said, they will have access to the map editor, so if they'd like to contribute with game design (and if I like their game design contributions), their role might expand. :)
Delix
My thoughts took me to the realization that I had an API problem; what is an API that could resolve both into something that looks up into a set of tags at run-time in release mode, but resolves to a constant at compile-time in developer mode?

I think you inverted "release mode" and "developer mode" in this paragraph.
mrmixer
I think you inverted "release mode" and "developer mode" in this paragraph.


Sure enough, thanks for pointing that out!
Wow, that's super cool! Good work, man.
Very cool looking! I had a question that popped up as I watched your video -- what's the overall benefit your system brings as opposed to letting someone overwrite the file on-disk? It seems like by overwriting the disk file all the operations your cool meta-program can do are still possible. Like switching to a different image (overwrite with a copy of another), or just do hotloading by editing the file.

I suppose one benefit is that if someone wanted to change image A referenced in game to image B, then without your system there would be two copies of B and A would be lost (by overwriting A). But is changing an image from one thing to another actually a common use-case? I imagine the most common feature needed would be to update an image, and not to update an image reference.

Just curious what your thought process was going from customer need, to a solution, as I seem to be missing something.
Hey Randy Gaul, thanks for the comment!

The system allows for both hot-reloading of the assets themselves, but also the asset tags (or references). A developer of the game can just modify the file on disk, and it will be reloaded by the game automatically.

Tags are reloaded to allow a non-programmer to modify the intent of an asset without having to contact me to do it for them. For example, if my artist would like to add a new type of object to the game, he can create a new tag and reference whatever filename he likes (it might be a new one, or an existing one). Then, when adding the object, he can specify the texture as whatever tag he specified previously, as opposed to the texture filename itself.

This system is useful because intent can be expressed irrespective of a direct connection to filenames, and intent for a resource changes less frequently than filenames do. Additionally, if one tag is referenced in many locations, modifying that texture's filename is not a problem. Another benefit is having multiple tags reference the same file, which allows a decomposition of intent. As an example (this would not exist in The Melodist), imagine having both "player" and "player_armor" tags, as opposed to requiring a direct reference to "player.png". Now, imagine a modification such that all armor art is moved to a generic "character_armor.png" file. With this modification in this system, all locations in the code (and game content files) that reference "player_armor" will not have to change. In the direct system, all locations in code and developer files that try to use the player's armor asset will have to be modified as well. If these are directly referenced in the code, the program must be recompiled to fit.