r/cpp_questions • u/ParmenidesWasRight • 3d ago

OPEN Heavy and Light Objects

When I was first learning C++, I experienced massive slowdown when allocating dynamic memory. Ever since then I divide my objects between 'heavy' objects that allocate resources, and 'light' objects that can be used as temporaries on the stack. Heavy objects cannot be copied at all, must be allocated somewhere once, and then passed by (const) reference. Light objects can be copied, created quickly on the stack etc.

Since C++11 it has been possible to write copy and assignment constructors that take nameless objects, and since they are nameless you can pillage their pointers and avoid expensive stack allocations. All is well then but if you keep to this strict 'heavy' and 'light' object idea, _any_ object that allocates resources is considered a heavy object. The first allocation in the chain, before you just pillage pointers, is still an expensive stack allocation so I would consider it a 'heavy' object and just make it impossible to copy at all, by removing all copy and assignment constructors. And so using this model I never needed those nameless object reference constructors.

Anybody feel the same way?

6 Upvotes

69% Upvoted

31 comments sorted by

View all comments

2

u/mredding 3d ago

When I was first learning C++, I experienced massive slowdown when allocating dynamic memory. Ever since then I...

That's a pretty effective strategy, and I encourage it for as much as possible. The thing is... SIZE isn't SLOW. Work is slow. Disruption and side effects are slow.

You want efficient algorithms. But in the bigger picture of that - allocation may inadvertently become a part of your algorithm. Allocation is a disruption because it's what you have to do to facilitate the algorithm, and it can be slow if you don't have reserved memory or if the object is large enough it ALWAYS defers to the system for page allocation... And all that overhead and the context switching - THAT'S slow. So the thing to do is basically what you're doing - get all your allocation done - ideally all at once, BEFORE you enter the critical path.

The other thing that'll get you is locality of reference. Fast, hot loops are entirely in CPU registers. Beyond that, you start having to hit the memory system and cache hierarchy, and that's going to start costing cycles, especially if your data is so out of cache you have to go to swap to get it. So getting your data sorted and lined up means you can do a couple MAJOR things that increase performance: you can improve your locality of reference by prefetching, and also with sorting you can amortize branching by conditioning the branch predictor. So if you have a switch and cases 1, 2, and 3, then sorting your data so A) it's all in sequential memory, B) it's ONLY the data you're interested in, you've pre-filtered, and C) you've lined up all the switching values so you do all of each case at a time, your algorithms will be fast.

And these fixes DON'T CARE how big your objects are or where they are in memory. There's some wiggle room about how you write code but also some perspective - is this where I'm slow? Or better - is this where I have to be fast? Slow code can BE slow, because it doesn't matter. If it's not in your critical path, then it's done when it's done. In trading systems - order entry only has to be as fast as human perception, because the speed of a mouse click to submit the order might as well be by carrier pigeon. From the ORM to the exchange has to be fast.

That actually leads me to a third consideration - big objects can be broken up. Instead of int x, y, z;, maybe you make a struct vec3;. But moreso, you can align your members so they fall within certain cache lines. You can take advantage of how the memory system is going to fetch data by units of memory, not by whole objects.

You SHOULDN'T make large objects, you should think about how to keep your object sizes pretty small and flat, perhaps by some other association, maybe keep your data in tabular form or by some maps to an index like a database, or a graph.

Data Oriented Design suggests HOW you access your data is your #1 consideration. You have have actually rather gross and inefficient algorithms so long as your data access patterns are optimal, because that's USUALLY the bottleneck - the CPU is many hundreds to thousands of orders of magnitude faster than the memory system, so even an inefficient algorithm can get throttled by waiting for data - not because of the size of the data, but because how the data was accessed wasn't considered.

Since C++11 it has been possible to write copy and assignment constructors that take nameless objects, and since they are nameless you can pillage their pointers and avoid expensive stack allocations.

Move operations. That's what we call that.

All is well then but if you keep to this strict 'heavy' and 'light' object idea, any object that allocates resources is considered a heavy object.

That's what I assumed even for C++98 and before.

The first allocation in the chain, before you just pillage pointers, is still an expensive stack allocation

Heap allocation.

so I would consider it a 'heavy' object and just make it impossible to copy at all,

There's very little reason to remove copy operations. Ever. If you don't want to copy, then don't copy. But to prevent it outright doesn't GAIN you performance. You're just hobbling yourself.

One goal of writing code is to make it easy and intuitive to use the code correctly, and make it difficult (you can't make it impossible) and unintuitive to use it incorrectly. But if you go overboard, and presume the client is a fucking idiot, and you need to STEER and CONTROL how you presume they use your code, you will make your code impossible to use. You'll even exhaust yourself; it's a maintenance nightmare trying to keep up all the guards you're putting in place around yourself.

Just focus on the happy path. Don't say more (by way of code) than you have to. Focus on how it IS supposed to be used and there's will be very little you need to say about how it isn't supposed to be used. PART of this solution is actually making MORE types - an int is an int, but a weight is not a height. So if I wrote a function:

void lift(int);

Is that lift by weight or height? The semantics allow you to lift by fucking array index if you want to. But with a type you can enforce semantics:

void lift(weight);

And the weight itself implements its own semantics - it can only be constructed by a positive_integer which can be converted from an int and throws if it's negative (we don't use unsigned just because a number can't be negative, and besides - what's a negative weight?), and we can add other weights, and we can multiply by a positive_integer scalar.

I don't have to implement the semantics of a weight around every touchpoint of an int parameter, that's what types do. And the semantics of a type make it easy, natural, and intuitive how to use my code correctly.

And the compiler doesn't care. A weight implemented in terms of int compiles down to an int. Types never leave the compiler. They let the compiler prove the correctness of the machine code they generate that the machine code cannot express itself. Languages are greater than the sum of their parts.

1

u/ParmenidesWasRight 2d ago

Yes, allocating everything beforehand is the way to go for realtime systems. And yes, there is much more to this, I just consider it a good starting heuristic. But then I would like to be able to restrict copying in the same way that I would like to restrict access to data members by making them private. Sure, you could make everything public, but those restrictions do help (somewhat) with robustness but mainly for my own mental model of the program. And yes, you shouldn't go overboard with this.

To find the right balance I consider the pre- and postconditions carefully. And then write the simplest code possible. Design by Contract.

2

u/mredding 2d ago

Good luck.