Yarnspin (Story-telling game engine)

image.png

Here is a sample game made with the engine (plays in browser): https://mattiasgustavsson.com/wasm/yarnspin

I have always been fond of game dev toolkits and construction sets. There is a beauty in the limitations they set, and something empowering about the simplicity they offer. I often enjoy figuring out how to make what I want to make within their given restrictions. When I first started using computers, first the Commodore 64 and later the Atari ST, I used a bunch of different gamedev tools.

devtools.jpg

Yarnspin as a concept, started out as the desire to make such a simple, oldschool engine. I imagined it as a choose-your-own-adventure game maker, and early on I intended to have features like money and skills, but later on I removed them to simplify things,

FW6UR8ZXoAY9bCF.jpg

From the very start, I had the idea to make it in a retro style, but I also wanted to see if I could build in some tools to let you use any pictures and have them automatically fitted to a common, limited palette. I wanted it to be easy for anyone to make a game with it. When I first started out making games, I often stumbled on not being able to get something that looked decent. I wanted Yarnspin to try and bridge that gap.

FW6USTgXoAkvBP6.jpg

The UI/layout was in part inspired by the games Maupiti Island and Mortville Manor on the Atari ST.

img3.png

I decided to show the characters present in a side panel, just like in those games, and to have a corresponding items panel on the other side. At one point, I considered compass buttons for navigation, but dropped it. I might bring that idea back, to free up some options slots.

FW6UTBJWIAIoy6E.jpg

When it comes to dialogs, I decided to keep the layout simple and similar to the locations screen. As you can only talk to one character at a time, there is no character list, but there is the item list on the right (empty in this screenshot).

FW6UTWqXgAELgmb.jpg

By changing the palette used, you can dramatically change the look. There’s a bunch of palettes included, and you can supply your own just by giving it an image file with the palette.

pal1.jpg

Using very limited palettes can give a highly stylized look. There is no emulation of graphics limitations beyond palette though. We are not trying to make things look like a specific old system here.

pal2.jpg

From the very start, I wanted games to be done by writing scripts, not by clicking a UI. Part of it was not having to build a visual tool, but also, I like working with text, I like the overview I feel I am getting with text files.

In fact, before even starting on the graphical parts, I had a very simple testbed for exploring the scripting language:

nedladdning.png

I wanted the scripts to be a lot more friendly than general purpose programming languages. You shouldn’t need to be an engineer to use this. My first version was in the form of ini files, which was very easy to implement, as I already had an ini loader. But it was awkward to use,

FW6UWo4XkAMTopn.jpg

I soon moved to a custom parser, and that let me simplify many things. This is what the final scripting language looks like. This is a location named “welcome”, which shows an image and a text. When the user clicks to proceed, it goes to the new location “your_desk”

FW6UW_3XwAAHGLv.jpg

Adding options to a location is done like this. opt: declares the text to display for the option, and act: specifies where to go if that option is selected. Also, in the first line of the script, a flag (visited_another_room) is set to true. Flags can either true or false.

FW6UXYPWQAI7UKp.jpg

Flags can be tested like in this script, and text or images can be included or omitted based on any number of flags.

FW6UXufXkAES9v4.jpg

The scripting language has some more features as well, but they are all in this style. There are no arithmatic or variables beyound flags (though this might change in a future version).

As for the code, the whole thing is done entirely in C, using only stb-style single-file C libraries. It is built in just a single compile unit, so I don't need any build system - you just do:

cl source\yarnspin.c

to build everything.

I thought I’d talk a bit about the code components making up Yarnspin. Just a high-level overview, a look at what libraries I’m using and why, and how the different parts of the code is organized.

I guess most people on here are already well familiar with stb-style libs, but I'll explain them a bit anyway (at least my take on them).

The full source code is here: https://github.com/mattiasgustavsson/yarnspin/tree/30a54a3fc245986e7b23e7147ac2b808567e03fd

Let’s start off by looking at the Yarnspin code files, before looking into the libs. They are only just over 6k lines of code. There are two separate applications in yarnspin. One is the compiler and runtime, the other is the image editing tool. The latter is all contained in imgedit.h, and the former is the other files:

image.png

As you can see, there is just one .c file - everything else is a header file. Yarnspin is built from stb-style single-file libraries, and with the small projects I do, it makes a lot of sense to put everything in a single compile unit. This is a pattern I use in all my projects.

So yarnspin.c contains the program entry point, main. And at the top of the file, I include all the libraries I am using - I’ll talk more about the different libs in a bit. The thing about stb-style libs, is that doing these includes only gets you declarations, not definitions.

image.png

This makes them different from C++ template header-only libs, which gets you both declarations and definitions at the same time. At the end of yarnspin.c, I include the same .h files again, but this time, each is preceded by an _IMPLEMENTATION define to enable definitions.

image.png

These defines work similar to include guards, and toggle on the majority of the code in the lib, the actual definitions. If I was using a lot of huge libs, and they were to cause long compile times, I could put this second set of includes in a separate .c file. Compile times is never a problem for my projects though, so I keep things to a single C file for simplicity and convenience. Compiling yarnspin.c, which includes all the libs, is pretty much instant.

Now, I could have added the _IMPLEMENTATION defines before the includes at the top of yarnspin.c, and then I wouldn’t need the second set at all. But then all the internal symbols and defines of the lib and files it includes would be visible in my app code and to intellisense. And that just makes things a bit noisy, so I prefer splitting things up like this. But having just a single set of includes for both declarations and definitions works just as well, and I sometimes do that.

The main function in yarnspin.c will look at the commandline parameters, and decide to launch the image editor or to compile and load a yarn and launch it in the game runtime Yarnspin use my own app.h lib which create a window and gl context, and calls your own app_proc func, where the app code is implemented.

image.png

image.png

app.h runs on Windows, Linux, Mac and in the browser, and compiles with gcc, clang, msvc and tcc. It can be built as either C or C++, and has a pretty streamlined API surface.

It would be nice to get around to making custom linux/mac backend implementations for app.h, but for the time being, I use SDL2 for linux and mac. For windows and wasm though, it has custom implementations that don't need SDL.

Now let’s have a brief look at all the libraries I’m using, and what I’m using them for.

image.png

I guess stb_image.h should be obvious - I use it to load images of any format, jpg, png, gifs etc. It’s an amazing file loader lib, highly recommended.

The next one, lzma.h, is the public domain LzmaLib conpression library by Igor Pavlov, repackaged into a stb-style single header lib by myself. I use it to compress the final data file for the game - more to make is content more obfuscated than because it really needs compressions.

stb_truetype.h reads ttf files, and renders them to pixels. I use this to convert ttf fonts to my own format for efficient rendering, as can be seen here: https://github.com/mattiasgustavsson/yarnspin/blob/f32201914cab601500b34945453d77bee0cbf8ca/source/gfxconv.h#L81. I’ll talk more about the in-game font renderer, pixelfont.h, in a bit.

The next lib (skipping app.h which we already looked at) is cstr.h, a fairly new and not entirely complete lib by myself for handling strings in C. It is mostly designed for ease of use and convenience, while still not wasteful in terms of performance. I’m still trying it out :)

stb_image_resize.h is used by both imgedit and yarnspin, to resize images from whatever input size to the fixed output size of Yarnspin. stb_image_write.h is not strictly necessary, but I use it to save a copy of each processed image as a PNG in the “cache” folder.

crtemu.h and crtemu_pc.h are two variants of my shader that emulates old CRT screens. What I really want to do at some point, is to implement the settings parameters in those libraries, and unify them into one lib with two default settings.

thread.h is my own library for doing multithreading - mutexes, threads, signals etc. It has implementations for Windows and Posix. It is used only by imgedit, to have a background thread which process images without blocking the main UI thread.

ini.h is a library I wrote to read and write classic old-school .ini files. I like the simplicity of those, and I use them whenever appropriate instead of XML or JSON. In Yarnspin, they are used to store the settings made with imgedit, so images gets processed according to those.

Yarnspin is limited to 256 colors, and the palette is loaded from an image file for simplicity. If the image contains 256 different colors or less, they are used as is. If there are more than 256 colors, it will use palettize.h to run a median-cut palettization algorithm on it.

paldither.h is used to take a 32-bit image and a palette, and convert it to an 8-bit palettized image using the specified palette and a choice of dither patterns. It is rather slow, as it loops through all combinations to find the best match. Really needs algorithmic optimization.

For drawing text in the game, I use my own pixelfont.h. It can draw with different alignment, wrapping, bold/italic/underline, even slow-print. I use it all the time, for all my text rendering needs. It has functions to create a font in the right format, from bitmaps of glyphs.

Once paldither.h have produced an 8-bit image, I use palrle.h to make a run-length-encoding of it. This is what’s saved to the output file, and the lib has functions to blit straight from the RLE data.

buffer.h is just a simple helper lib for reading/writing binary data to/from a dynamically growing buffer. It is quite handy when implementing custom binary formats. It’s used like this: https://github.com/mattiasgustavsson/yarnspin/blob/f32201914cab601500b34945453d77bee0cbf8ca/source/yarn.h#L150

By default, yarnspin does a bunch of adjustments to images before palettizing them, things like increasing contrast/saturation and applying a ”sharpen” filter. With imgedit you can set how they are applied. The img.h is a wip lib I use for doing that kind of image manipulation.

array.h is another new lib, for handling dynamic arrays in C. It works reasonably well, but is a bit lacking when it comes to type safety.

dir.h and file.h are simple utility libs for listing files and loading/saving them.

sysfont.h is a fixed font renderer with embedded font data, used only by imgedit to not have to rely on a specific font file being present.

So there we have it, all the libs used in yarnspin :)

I am currently working on version 2 of Yarnspin, and it has a lot of new stuff like sound/music and full hd RGB support. Most of it is ready, and release is not that far away. I might talk about that in a future post.


Edited by @Mattias_G on

I have always been fond of game dev toolkits and construction sets. There is a beauty in the limitations they set, and something empowering about the simplicity they offer. I often enjoy figuring out how to make what I want to make within their given restrictions. When I first started using computers, first the Commodore 64 and later the Atari ST, I used a bunch of different gamedev tools.

I have very good memories of trying to figure out things in the Duke Nukem 3D editor, with little documentation, and limited knowledge of English. It was a game in itself.

I'd like to give yarnspin a try when I get some time.

In the tutorial game, at some point Alice says something like "...to keep track of what have happened...". I think it should be "has happened" (I'm not a native English speaker so I might be wrong).

While looking at things I found No sunshine, but it seem to set its size based on the width of the navigator window, which makes the bottom of the screen outside the view and require to use the scrollbar (A workaround is to resize the browser window to get a view without scrollbars).

The lack of sound and music was a bit weird, but you're already adding that in the next version.

Good luck

Thanks :)

Yeah, No Sunshine was using an older html template, and it wasn't scaling correctly, but I have now updated it, so should work better - thanks for pointing it out :)

I've made a game with yarnspin: YarnShip

Sources

It's probably too obtuse for most people to figure out, but that's what I do. And I "miss-used" the engine.

It ends abruptly as I cut half of what I wanted to do (and you can die if you mess up).

One issue I have with the engine is the messages when flag are tested but never set (or set and never tested). Those were super annoying when testing things as disabling one thing would require me to disable other things and then re-enabling them.

I also had confusion about the "scope" of flag tests. I often wanted to do several thing after testing a flag, which doesn't seem to work. But for "opt:" it worked (if I remember correctly). I would have liked to define scope using empty lines.

flag ?
    act: set a
    act: set b
    act: set c

// instead of
flag ? act: set a
flag ? act: set b
flag ? act: set c

The source contains some unorganized notes in main.txt (mostly thing that I didn't understand or still don't understand).


Edited by Simon Anciaux on Reason: Updated links

Wow, that is awesome :) I think it's the first example I've seen of someone else using this engine to make a game. It is super useful to see, and the notes you put in main.txt is incredibly helpful! It will help me a lot as I continue working on documentation, and I will likely make some improvements to the engine as a direct result of it as well. So thank you so much for sharing this!

I enjoyed the game - I really enjoyed the way it presents a puzzle where you have to figure out how to interact with the controls of your ship. The engine was obviously written with story-telling in mind, but I would not have thought of telling a story in this way, I thought it was great. And the art was great too.

As for the flags giving error if tested but not set, it is meant as a way to avoid typos for the flag names. There is the option to explicitly declare flags though, like "flags: myflag, myotherflag" etc, and that disables those checks, but instead gives errors if you use a flag that is not explicitly declared. I need to document this better.

I can see how scope of flag tests comes across as confusing. A test only applies to the next thing, but what "the next thing" means is slightly context dependent. If you have a test before an act/img/txt statement, it only applies to that statement. But, if you have a test before an opt/use/say statement, it applies to both that statement and the act statement that goes with it.

Introducing some way to define scope is an option, but I am somewhat reluctant to do so, as I feel it complicates some aspects of the scripting language (while admittedly simplifying others). For the version after the one I'm working on now (so v3, whenever I start on that) I have some loose plans to include integer values/calculations/expressions, and maybe I'll need to revisit scopes for that anyway.

Again, thanks for the great feedback, and for trying the engine!


Replying to mrmixer (#29590)

Glad you enjoyed it.

There are two features that would be nice to have:

  • saves, both user saves and scripted saves (useful for development and playing). It was a pain to have to go through the game several times to test things (even if the game was small). I could have tried set some flags and go to the correct section but I wasn't confident enough that I wouldn't miss something.
  • a way to reset the game (clear all flags and go the entry point). In my case, if you fail and die, I choose to exit the game as there are a lot of flags in the game and I didn't feel like clearing them all by hand. Exiting the game was confusing to the few friends that tested the game (I could make that clearer though).

But those needs might be related to the type of game I made.


Replying to Mattias Gustavsson (#29606)

Yeah, agreed, saves are already in for v2 (currently not released on itch, but can be built from source from github). There's user saves (for both web and native builds) with 9 save slots, and also quicksave/quickload commands which can save/load programatically, but only has a single slot.

There is also a new debug mode which allows you to set starting flags and sections to enable debugging from known states.

There's also a new restart command that starts over.

Oh, and it's much easier to package the game for distribution, you just go yarnspin --package game.exe or yarnspin --package game.html and it will make a single file that contains everything for native or for web.

I realize now that I never shared the plans for v2 here. But I have this writeup: https://mattiasgustavsson.itch.io/yarnspin/devlog/522668/yarnspin-v20-dev-update


Edited by @Mattias_G on
Replying to mrmixer (#29608)

so here's some news finally :) I have decided I am close enough to share an alpha build of Yarnspin 2.0. Of course, anyone building from source will have already have access to everything as I go along, but now the only things left are some script validation things and finishing the documentation, so I figured I might as well share an actual pre-built version of it as well. If nothing else so I can check that my automated "build, package, and publish with github actions" scripts work as they should.

Builds can be downloaded here, for win, mac, linux: https://github.com/mattiasgustavsson/yarnspin/releases/tag/v2.0.alpha

There's one new feature I want to specifically highlight. Something I've been wanting to do for a while, and now it is finally in place. It is a means of packaging your game for distribution. You can now do

yarnspin --package mygame.exe

and get a single, self-contained executable file containing all code and data for your game - you can simple distribute mygame.exe and that is all.

But also, you can go

yarnspin --package mygame.html

and this will produce a single, self-contained html file that contains everything for your game. You can share this file any way you like, upload to itch or your own site, and it runs in browsers, and on phones too.

And these things work (or should work, most of the testing so far has been on windows) on all platforms, without having to install any tools, just unzip yarnspin and run it

I tried to "port" my game to version 2, but there are a few issues.

  • What resolution should the faces be ? The old 90*90px is processed differently in the new version and looks very bad. The original art doesn't need to be processed as I only used the color from the palette I created, so I'm assuming it's an issue with the resolution;
  • The "font directives" aren't present anymore (it seems) but they are still present in the documentation PDF;
  • I would make the "version" directive optional and default to 1 if not present;
  • I like that there isn't a requirement for a logo anymore at the start, but it's still possible to do if I want to;
  • The characters "blinks" which is attention grabbing (that's the point I guess) and I don't want that in my game (so I would be nice if it could be an option to turn on or off);
  • I also don't want the "You have nothing" text (I'm assuming there is a thing similar to "alone_text:" but I don't know what it is);
  • I tried the two package commands and they didn't work. The web one crashed, and the exe one displayed "Failed to copy build\runtime.exe";
  • After building from the source and copying the build folder next to the yarnspin exe, the exe packaging works;
  • The wasm build tells me that "build\node" isn't a command or executable;
  • What would be nice (and what I expected) is that with one download I could package the executable and the web version.

Replying to Mattias Gustavsson (#29738)

Thanks for trying it. I finally got some time to get back to this.

  • Face res have been upped from 90x90 to 112x112, and yes, if you make them that size and only use colors from the palette, they will not be processed
  • The font directives have been renamed, but the documentation is not yet updated. The new names are font_txt, font_opt, font_dialog, font_say, font_response, font_chr, font_use, font_name. And you can also optionally add a size specifier "font_txt: myfont.ttf, 23"
  • The version is used for savegame versioning, to make sure not to try to load old savegames, the idea being that when you make a new release that would break old savegames, you up the version (this would also be explained in the docs, when I do those), so I think it is better for visibility to not make it optional
  • I am glad you like the replacement for the logo, I think the new "screen" concept works better, and it also fits better with playing music. and, of course, can be used to display full screen images anywhere in a game too
  • Good point about the blinking, I will make that configurable (maybe make it so you can turn it off completely, have it happen only the first time something is displaced, or happen all the time, and maybe separately for items/chrs). But yeah, the reason for it was that some players missed that those things could be clicked.
  • There is a "nothing_text" corresponding to "alone_text", but again, there's nothing in the docs about it yet
  • Packaging was not working as well as I wanted it to, but I have now implemented it differently (there's a new build, same link as before). Can I ask what platform were you building on? You are correct in thinking that all you need is the one download and it should build executable as well as web with no additional downloads or dependencies

Again, thanks for the feedback, it is so useful!


Replying to mrmixer (#29746)

Thanks. I've tried the new builds and the packaging works for both windows and wasm. And with the new "knowledge" I was able to fix the remaining issues.

The web version

The windows version

The sources

There are two small issues that I remembered while testing:

  • My game has 10 items in one of the location (numpad.txt), and while there is enough space to display them, I had to add a fake item so that the top item isn't clipped at the top of the window. I added ~ as an item because it becomes invisible (probably over the top of the screen) and can't be clicked.

  • The game menu at the top right "hitbox" (white triangle) is bigger than what we see. Mostly in the same location with 10 items, when I clicked the top item somewhat close to the game menu button, it opens the menu, which is annoying. The menu button extends to the center point of the items button horizontally which seems overkill.


Edited by Simon Anciaux on
Replying to Mattias Gustavsson (#29924)

That's great, thank you :)

As for the first issue, there is now some new options for controlling the look and layout. Examples can be seen here: https://github.com/mattiasgustavsson/no-sunshine-hd/blob/main/scripts/styling.txt (it is for a remake of the No Sunshine game, which is styled quite differently from the default https://mattiasgustavsson.com/wasm/no-sunshine-hd)

You should be able to use the global directive vmargin_use: to specify how much to offset the item list vertically, so you don't need the invisible dummy item.

As for the second issue, yeah, you are right that the menu button hit zone probably should not be that wide - I'll make it a bit smaller. The reason it is bigger in the first place is to make it easier to hit on small screens like phones. I just never had that many items, so hadn't noticed that it could be an issue.


Replying to mrmixer (#29928)

I didn't know about all the styling variables.

As I was looking through the example you displayed, I though it could rid me of another work around which was lining up options.

In the game I want to display some of the options (in a single location) to the left, right or centered to illustrate where thing are in the cockpit of the ship. To do that I had to use a dot followed by spaces (if I don't start with a dot the spaces are skipped).

Unfortunately the styling options are globals and I can't change them for each options. But finding workaround was part of the fun while making the game, so it's fine.


Edited by Simon Anciaux on
Replying to Mattias Gustavsson (#29929)

Yeah, I guess there's a balance between keeping complexity down and providing power/flexibility. I noticed the way you use dot and space to control alignment of options, so one thing I've already decided to add is being able to prefix a text/option line with a backtick ` which would then not be printed, but it would keep any spaces and not trim them off.


Replying to mrmixer (#29930)

hey, I wanted to ask you, would it be ok if I share your Yarnship game on the Yarnspin discord server? (https://discord.gg/ZVuX2pwS24)

I think the others there would find it interesting.


Replying to mrmixer (#29930)