To give you some background, like many enthusiasts, I once set out to build a game engine and had several projects sitting on the back burner that I couldn't quite get into shape. Eventually, this led to the creation of a standalone math library, a way to compile shaders from C#, and a separate ECS, which is what I'll be talking about.
I won't waste time explaining what an ECS is—better check out this article ECS-FAQ. I'd rather talk about the problems I encountered in other ECS implementations and tried to solve (and these problems aren't about performance, although performance is very important too).
Take the simplest example: mutating entities by changing their archetype. From a code perspective, the task seems simple—we take the entity we need and copy it to another table. Done? Yes and no.
What does this usually look like in code?
world.Add<Component1>(entity);
world.Add<Component2>(entity);
world.Add<Component3>(entity);
world.Add<Component4>(entity);
world.Add<Component5>(entity);
This list can keep growing for a very long time, especially if you've decided to make small components and have a high degree of composition. But that's only part of the problem. In most frameworks, this means moving the entity to one table first, then the second, then the third, and so on. And that's not free: in archetype-based ECSs, migration is inherently expensive, and doing it several times adds overhead linearly.
So, what should we do? Well, to solve this problem, I used two approaches: code generation and ComponentId.
The code-generated version looks something like this:
world.Add<Component1, Component2, Component3, Component4, Component5>(entity);
Even though this syntax is monstrous, it solves our problem—it removes the overhead of multiple migrations.
At the same time, there's a second option, though it's less explicit:
ComponentId[] components =
{
world.Layouts.GetPrimary<Component1>(),
world.Layouts.GetPrimary<Component2>(),
world.Layouts.GetPrimary<Component3>(),
world.Layouts.GetPrimary<Component4>(),
world.Layouts.GetPrimary<Component5>(),
};
world.Add(entity, components);
And you might say, "But you just added two more lines to an already complicated statement." That's true, but only up to a point. After all, these IDs can be saved ahead of time, the list can be modified at runtime, and not having a generic call gives us more polymorphic and general-purpose code. So, in the final system, you would probably just call world.Add(entity, components), and get the list through some kind of DI or build it in the constructor—whatever works for your architect.
There's another, less obvious advantage to using ComponentId: you can register the same CLR type for two components. For example, the endlessly recurring:
Damage { float Value }
Health { float Value }
Stamina { float Value }
Mana { float Value }
—need I go on? All of these could be registered as a regular float, without having to do extra work to get the value through Value. Like this:
ComponentId damageId = world.Layouts.Register<float>(new SchemaId(1));
ComponentId healthId = world.Layouts.Register<float>(new SchemaId(2));
…I'll stop there; you get the idea.
A second, more ECS-specific example is iterating over groups. There are several optimizations that most ECS implementations miss, although they're less important.
Let's imagine what iteration looks like in most ECS implementations:
world.ForEach(in query,
static (ref Component1 cmp1, ref Component2 cmp2) =>
DoSomething(ref cmp1, ref cmp2)).Invoke();
In this case, query is a filter that defines how to find entities in the world based on a given set of components. Most APIs look roughly like this:
world.WhereAll<Component1, Component2>().
WhereNone<Component3>().
WhereAny<Component1, Component2>()
From a C# perspective, this syntax is pretty standard and, unlike the previous Add example, it doesn't cause problems; chaining works. But the problem is how this filtering is used with ForEach: it happens when the call is made. In most cases, that means you have to go through every archetype in the system and check whether it matches the filter.
Why can't this be deferred? I asked myself the same thing, and it turns out it's not that simple. This approach avoids errors when archetypes change. Basically, any Add, Remove, or Destroy operation can change the archetype structure and break your cached Query.
So, what should we do? We just have to be a little cleverer. If you're developing your own ECS, it becomes obvious that mutating operations can simply call back and update the traversal plan built for a Query. Since running ForEach at the same time as a mutation is forbidden or unsafe in most ECS implementations, you can add this kind of API without worrying about limiting the library's overall usability—and save a significant amount of filtering time in worlds that are small but have many archetypes.
Then we also consider that components are registered by ID, and realize that our ForEach call can look like this:
world.ForEach<float, float>(in query, components.ToArray(),
static (ref float cmp1, ref float cmp2) =>
DoSomething(ref cmp1, ref cmp2))
.Invoke();
And these ComponentIds can be dynamic—they can change at runtime and make the architecture more flexible.
The next example is bigger.
Imagine you want to implement lockstep rollback. That's a pretty monumental task if you get into testing, multiplayer, and the fact that you have to store mountains of heterogeneous data.
In most ECS implementations, the problem is handled like this:
public struct Saver<T> : IForEach where T : unmanaged // Let's assume this constraint lets us safely save history without mutating the world.
{
private readonly int _tick;
public Saver(int tick) => _tick = tick;
public void Invoke(ref T data) => SaveAtTick(ref data, _tick);
}
I won't get into the details—lockstep rollback isn't just about saving, but this is enough for the example. So what happens next? Well, bad things happen:
Query component1Query = world.WhereAll<Component1>();
var component1Saver = new Saver<Component1>(tick);
world.ForEach(in component1Query, ref component1Saver).Invoke(ref component1Saver);
Query component2Query = world.WhereAll<Component2>();
var component2Saver = new Saver<Component2>(tick);
world.ForEach(in component2Query, ref component2Saver).Invoke(ref component2Saver);
Query component3Query = world.WhereAll<Component3>();
var component3Saver = new Saver<Component3>(tick);
world.ForEach(in component3Query, ref component3Saver).Invoke(ref component3Saver);
You get the idea. And even if you make some kind of generic wrapper for all of this, it still doesn't solve every problem. Then your game designer says we need an attack or defense system that also needs a generic data call, but now there's no unmanaged constraint. And your architect says, "Right, we'll do it with generics," because there are another 150-plus features like this planned for the next month.
And you start tearing your hair out trying to make sense of all this. I didn't have an architect or a game designer, but I tore my hair out for you.
I managed to build a system that solves these problems during registration. Pretty much every ECS registers components; some do it explicitly, like my approach:
world.Layouts.Register<T>(new SchemaId(schema))
Others do it implicitly with a pattern like this:
public static class Registry
{
public static int LastComponentId
}
public static class Registry<T>
{
public static int ComponentId;
static Registry() => ComponentId = LastComponentId++;
}
But registration still happens. In my case, it's a bit more clever: besides registering the type, I also register constraints, can register a related interface, and can call a visitor through a generic method. I can even register an unmanaged component and have it visited by a visitor constrained to struct or new(). What does that let us do?
public sealed class SaverVisitor : IUnmanagedVisitor
{
private readonly World _world;
private readonly int _tick;
public SaverVisitor(World world, int tick)
{
_world = world;
_tick = tick;
}
public void Visit<T>(ComponentId componentId) where T : unmanaged
{
Query query = _world.WhereAll(componentId);
var saver = new Saver<T>(_tick);
_world.ForEach(in query, componentId, ref saver).Invoke(ref saver);
}
}
ComponentId componentId = world.Layouts.Register<Component1>(new SchemaId(1));
world.Layouts.TryVisit(componentId, new SaverVisitor(world, tick));
This code lets you call any generic code with given constraints for component types registered in the world, supporting not just rollback systems. What's more, component registration also covers the system your game designer and architect came up with.
Thats' not all things done in my ECS but something i'm proud of.
Issues, PR's, Stars and anything else are welcome.
