So I have this data-oriented-ish entity-component system.
The purpose of this is to be able to define entities entirely through their properties, and being able to assign or remove properties at runtime.
I'm running into problems with my current implementation and I am thinking about reworking the entire thing.

And since Casey has started tackling this topic on HMH the timing feels quite appropriate.

An overview of the current structure.


There's the notion of a "layer" which acts as the entity manager and gets a bunch of component managers assigned to it. The component managers all use a base struct that contains general information, like mapping entities to component indices or "instances".
Component instances are then iterated through various arbitrary systems that are nothing more than functions. These are the backbone of the program, performing all logic, draw-calls etc.

What works well so far :)
Since the data is packed together it's easy to process in the systems.

What doesn't :(
Interactions between components of different types. If an entity has more than one component chances are high that those components are supposed to affect each other in some way.
It's basically the same issue that Casey described with the "Act-React" model. A lot of querying has to be done in order to find out if an entity has a certain combination of components.
Performance aside, this adds extra complexity and code to the systems processing the data.

So...
While I don't understand how Casey will implement his entity system, the idea of putting all properties in one struct made me think about the relationship between components of different types.
It seems to me that most cases will be about sharing properties, or one dictating the properties of the other, basically transferring of data.
So if the properties shared the same data in the first place then no additional relationships would need to be implemented in the systems.

The very rough idea for the new system.


In short...
Component managers are reduced to the functions they used previously to manipulate the data. Entities are used as the key for locating that data.
The function usage implies that the entity "has" the relevant property.
Since functions have access to all properties they can override operations done before them, or blend for that matter.

So what's the problem?
I don't really know how to implement this, let alone I'm not sure if this approach is feasible.
The biggest question-mark is mapping entities to their properties, and vice versa. This could potentially result in way more queries than what I have currently, but I'm not well read into techniques like hashing and lookup-tables so I imagine that there must be some way of solving this effectively.

To conclude this long-ass post, I'm hunting for ideas. Any topics I should read into? How would you approach an entity system of this kind?

My answer to this is,

what *exactly* are you trying to solve?

By which I mean, are you building a game currently? Are you finding that a simpler system is not doing the job for you in some way? In what way is it not working?

I have shipped complete games that used much simpler things than you are proposing, and they worked just fine in terms of functionality. (The one beef I have with them is the performance of various group operations on many entities at once, so in my new systems, that is the very specific issue I am designing to solve).

I personally have never needed many vague components of unspecified functionality that affect each other in unspecified ways. If that is your problem statement, then of course you aren't going to know the answer, as there is lots of "vague" and "unspecified" in there.

In all these matters, I recommend keeping the design reality-based.

jon
My answer to this is,

what *exactly* are you trying to solve?

By which I mean, are you building a game currently? Are you finding that a simpler system is not doing the job for you in some way? In what way is it not working?

Good question. Seems like I haven't thought that through as much as I imagined.

In short, no there isn't really a game right now, I have something in mind and I have some test cases of rudimentary systems that I want/need.

In my mind the new approach I'm suggesting is simpler than what I currently have but it wouldn't surprise me if I'm still over-thinking things.

What I've found is not working very well is that the relationship between components has to be explicitly stated in the systems. If an entity has component A and B then X should happen. I imagine it would be much simpler to design the game if the components drove the functionality and the functions have the relationship hard-coded into them.

Probably not a satisfactory answer to your question but hopefully it gives some clarity.

jon

I have shipped complete games that used much simpler things than you are proposing, and they worked just fine in terms of functionality. (The one beef I have with them is the performance of various group operations on many entities at once, so in my new systems, that is the very specific issue I am designing to solve).

Interesting. What kind of group operations? Like gathering entities that has the same properties?

jon

I personally have never needed many vague components of unspecified functionality that affect each other in unspecified ways. If that is your problem statement, then of course you aren't going to know the answer, as there is lots of "vague" and "unspecified" in there.

Makes sense, but I don't know if that applies to my current situation. The components are quite specific and so is their functionality. The problem I'm trying to solve is how relationships (or connections if you will) are handled between them.

jon

In all these matters, I recommend keeping the design reality-based.

You've certainly given me some food for thought. I thank you for that.

Remember what Casey is saying all the time on the stream - write the usage code first. So first write game code that is supposed to use your entity system, instead of trying to figure out how entity system should look like.

Yeah, this really can't be overstated.

The history of software is full of people who wanted to write a system to do a really great job at random thing X (like, say, being a game engine with really cool entities or something). And these pretty much always turn out not to be good. The reason is that they are too divorced from reality.

Why do you even want components? I have never actually made a game that used components (in part because back when I worked at bigger companies on codebases that used components, I strongly disliked how complicated they made it just to do simple things).

If components are causing you to not know what to do, get rid of them. Maybe later, when you find that you really need them to solve a specific problem, you can bring them back. Until then, don't bother.

The system I use in both Braid and The Witness is a very simple single-inheritance system, where there is a base class Entity that goes like this:



and a bunch of subclasses like Door that go like this:



Of course it gets more intricate than this, most specifically the process of initializing and registering entities in the world and making it so that you can query them quickly, etc. But the data layout is EXACTLY this. It is not anything complicated ... and the number of copies sold of these games is in the millions, and we built them with small teams.

I would encourage you to go back and watch the Mike Acton video where he talks about understanding your specific problem and then writing code to solve it. That is what you want to do. If you don't have a specific problem, then I don't think designing an entity system is particularly productive. (Not even as a "just for fun" thing -- it is too much of an open, ungrounded, vague problem).

The components are quite specific and so is their functionality. The problem I'm trying to solve is how relationships (or connections if you will) are handled between them.

What are the components? If you say stuff like "rendering transform", I would ask: why are you making that a component? What problem does it solve to make that a component?

Interesting. What kind of group operations? Like gathering entities that has the same properties?

For example, when we want to autosave the game, we need to scan all the entities for changes. The Witness shipped with something like 60k-70k entities in the world. On the PC this was pretty fast but on the PS4 it was introducing frame hiccups (because it is a lot easier to get blocked on reading memory on a console), so we did a lot of annoying things to make it go faster, which complicated the save code really a lot, and reduced confidence that it was really correct any more. If it were faster to scan through the entities for changes, *or* to just copy the entity data en masse off to a separate thread that could do the job of autosaving, then there would not have been such a problem.

What you are doing is a "solve all" design, which is unimplementable.

I have never published a game, but I have published many projects.
Now what you designed is maybe something I would take to a board meeting, it is a great top down view of an enterprise project.
However it is very impractical for development, because during dev. you make so many changes and find so many ways to improve that starting with a full high level design is just a waste of time.

2 cents

Just popping by to say that since last time I've gone back to the drawing board, done some thinking and now I think I got a better grasp of where I should put my priorities.
What I took away from my experiences so far was that I put too much focus on structuring the data and not on the usage cases, which is probably why I came to a stall. I also got sidetracked with various things like trying to build a level editor which I thought I needed for solving particular data-structuring problems I had which turned out it didn't.

So since then I've been working on my new entity system, roughly the same concept as the one I said I was leaning towards, getting rid of "components" and "layers" entirely. Now it's currently just one big struct of arrays and two matrices for mapping entities to indices, much simpler.
A also failed to mention before that all the logic in the game will be driven by behaviour trees. This is quite a huge part of the entity system since that's where most of the usage code actually goes, which I completely neglected for a couple of months while getting sidetracked. Lesson learned there.

So basically I'm more pleased with how things are turning out right now and I'd like to thank you all for being straight forward with your advice.

Off to make new mistakes!

Great to hear of your progress, Andreas! :)

Well, I am glad we did not discourage you from continuing due to negative feedback!

I recently started a project to make a text adventure 'engine' in C++ while following some of the high level ideas I've been learning about from guys like you, Mike Acton, Casey etc. I was avoiding inheritance but at one point just said "no damnit, this all gets much simpler if I have a base entity that all these things derive from!". So I was amused to see this post where you do exactly that! Do you keep the depth at 2, or did you ever find it advantageous to go deeper than that?

I don't ever do a deeper depth. There isn't a good reason.

Hi,

almost four years ago I was hired by a company to make a mobile simulation game for iOS (in ObjectiveC).
I was new to the field so I asked around what kind of architecture I should use.
Some people pointed me to Entity Systems and there was even a small library which a co-worker already wrote.
It was a port of something he had used in his previous company, in ActionScript3.

The lib had only 3 final (non inheritable) classes. Entity, Repository and a Group. And it had 2 interfaces/protocols Component and System.
An Entity was just a wrapper around a hash map, where key is the Component type and the value is the component instance.
Component type is a class which implements the Component interface.
Other wise Entity had only AddComponent, GetComponent, HasComponent and RemoveComponent methods.
You can only create and destroy an entity by calling create and destroy methods on Repository.
You can also ask the repository to return you a group of entities which all have a defined set of components.

Now to the systems. They are classes which implement System interface. System interface has only one method `execute`.
This method is supposed to be called on every tick. A system queries the repository for a group or multiple groups of entities and operates on this group/groups.

Now to compare it to what Casey presented in episode 277. Repository is this sparse matrix. Let's say entity is a row and a component type is a column.
A group is a View on this matrix which will return you only the rows which have values in the given columns. The rows are "compressed" through the hash map.

Now we had a couple ports of this library. And the latest and most powerful is in C# made for Unity3D
https://github.com/sschmid/Entitas-CSharp

In the C# port we actually doing almost the same thing what Casey suggested in episode 277. The entity class is now backed by an array which has the length of number of component types. So every component type has an according index. We abandoned hash map because we traded memory footprint for speed.
As C# is a garbage collected language all the entities and component instances are pooled. We implemented code generator which hides pooling and provides convenience methods on the entity class.

Now coming back to your case. I think the man problem that you are facing is that you can't query for entities. You have to scan through all of them to find the ones that your system is interested in. Have a look at our Pool (we renamed Repository to Pool) and Group class implementation. If you are not familiar with C# you can find links to other language implementations on the github repository. There is also a C++ port.

I hope I could help in some way.

Jonathan Blow

and a bunch of subclasses like Door that go like this:

1 2 3 4 5 6 7

struct Door : public Entity { float open_t; float open_t_target; char *open_sound_name; char *close_sound_name; .... }

I'm curious if these structs are passed around as data or do you have methods attached to them, along with all the copy constructor and other stuff needed for copying objects?

Thimothy

I'm pretty sure it's just a flat array of Door*. There is no reason to copy a door struct after the game has loaded.

I remember Jon complaining about all the extra copy ctor() stuff in a Jai rant, so I thought it might be because of this.
Also, I'm just curious if these are really just structs or public classes with methods.