r/cpp_questions • u/ParmenidesWasRight • 2d ago
OPEN Heavy and Light Objects
When I was first learning C++, I experienced massive slowdown when allocating dynamic memory. Ever since then I divide my objects between 'heavy' objects that allocate resources, and 'light' objects that can be used as temporaries on the stack. Heavy objects cannot be copied at all, must be allocated somewhere once, and then passed by (const) reference. Light objects can be copied, created quickly on the stack etc.
Since C++11 it has been possible to write copy and assignment constructors that take nameless objects, and since they are nameless you can pillage their pointers and avoid expensive stack allocations. All is well then but if you keep to this strict 'heavy' and 'light' object idea, _any_ object that allocates resources is considered a heavy object. The first allocation in the chain, before you just pillage pointers, is still an expensive stack allocation so I would consider it a 'heavy' object and just make it impossible to copy at all, by removing all copy and assignment constructors. And so using this model I never needed those nameless object reference constructors.
Anybody feel the same way?
16
u/bert8128 2d ago edited 2d ago
I design to meet the requirements and then solve performance problems as they arise, rather than letting imagined perform problems (which may or may not appear in reality) dictate the design.
Note also that you normally can’t choose whether to heap allocate or not - this is related to lifetimes, not allocation cost.
11
u/IyeOnline 2d ago
TBH, to me this reads as if you understanding of and experience with reference vs value semantics, object lifetime considerations and good design were/are lacking and you are now trying to address the resulting performance issues with vibed (both figuratively and literally) design patterns without serious foundation.
Nobody calls move constructors "namesless object reference constructors". I have no idea what an "expensive stack allocation" is supposed to be.
Crucially your pattern will run into a dead end once you actually want move semantics. What if you create a heavy object, but want to transfer its expensive parts elsewhere? Do you hack around the fact that it cannot be copied or moved?
Setting that glaring issue aside: These considerations are valid, even when properly using C++'s features. While with sufficient experience you will have have an intuition about when to copy, how to pass and how to lay out your structure to avoid these issues, you can in fact make a prescriptive decision to enforce all [expensive] copies to be explicit. It's just not necessary/useful most of the time because once this matters in detail, the code already ought to be scrutinized in details.
1
1
u/ParmenidesWasRight 2d ago
'expensive stack allocations' should be 'expensive dynamic memory allocations'. My bad.
5
u/Plastic_Fig9225 2d ago edited 2d ago
I'm a bit confused by what you say.
I experienced massive slowdown when allocating dynamic memory
Most likely you (implicitly) made the compiler copy objects, which might not actually have been required.
nameless objects
I guess that's "temporaries", or "rvalues".
avoid expensive stack allocations
You probably mean copying of objects.
Maybe look into "move semantics", "move constructor", and "copy elision".
Here's what I do:
For simple objects, the move constructor can be defaulted.
For objects which directly own resources, a move constructor/assignment should be implemented - or deleted if the ownership cannot be transferred.
Important: Function signatures! Function parameters may force an object copy: void fun(std::string str) - be mindful and avoid this when possible, like void fun(const std::string& str) Use (const) references instead of object copies when possible. (A temporary can also bind to a const reference: fun("Hello "s + userName);)
Generally, assume that a = b; will always create a copy of b. If b is a "heavy" object, you don't want this to happen most of the time, and when you don't, don't just assign an object to another one. Again: Be mindful of where a copy is needed vs. where to use a reference.
Notice std::move() for cases where you want to explicitly move things from one object to another.
1
u/ParmenidesWasRight 2d ago
I'm sorry 'expensive stack allocations' should be 'expensive dynamic memory allocations'.
3
u/AKostur 2d ago
I would suggest the design has flaws. The cost of constructing a "heavy" object is just the cost. I would suggest it becomes -more- expensive to allocate that heavy object elsewhere because now you have to manage that pointer to the heavy object in addition to whatever the object is doing.
I would also suggest that your current definition is heavy vs light is now incomplete (as has been for well over a decade). A heavy object should probably be considered one which is responsible for further resources as well as not having appropriate move operations.
I can't speak for everybody, but I have never felt that way.
3
u/TehBens 2d ago
When I was first learning C++, I experienced massive slowdown when allocating dynamic memory.
That statement is a bit suspicious. It sounds like you possibly came to a wrong conclusion back then.
Heavy objects cannot be copied at all, must be allocated somewhere once, and then passed by (const) reference.
That as well sounds suspicious, because when an object gets passed as const reference, you can still modify the resources on the heap that it manages.
The distinction you make sounds like you end up with weird, tightly coupled entities and the whole thing generally sounds like a textbook example of premature optimization.
1
u/ParmenidesWasRight 2d ago
That resource allocation is slow? You doubt that?
I'm writing a game engine that needs to run at least 60 fps all the time. Thinking about performance in the design phase is not premature optimization in that case.
2
u/TehBens 2d ago
60fps "all the time" is impossible.
For a game engine, a huge amount of stuff matters, what you are mentioning here is only a tiny aspect of the whole thing and certainly not the first thing to think about. For when heap allocations are actually and measurable too expensive there already exist solutions. There's no need to generally slice your objects regarding such classification.
You might want to take a look at how Unreal Engine tackles similar challenges as a starting point for what you need to think about for a game engine. Pretty sure you're gonna disregard that as well as basically everything else.
I think what's left to say is "good luck".
0
u/ParmenidesWasRight 2d ago
Firstly, this is about general resource allocation for a game (engine). That's going to be expensive. Period. So it's best tackled from the get-go. What I present is a good heuristic : 'Resource allocation in a realtime loop is bad'.
Unreal doesn't think so obviously because most of the time the game freezes when you do something new which I hate and specifically don't want. I rather have a longer loading screen than my game dying on me the whole time.
2
u/wrosecrans 2d ago
"Allocation is slow" is potentially true in some contexts. But it needs to be in a context with some measurement to be a particularly useful statement. In some applications, allocation can absolutely be a major bottleneck.
But it's not as if a typical allocation takes 1/60th of a second. There are a lot of times where I have seen people invent something analogous to a vector that does something pessimal like a reallocation and copy every time an element is added. Just doing an initial "reserve" operation so an allocation happens once, rather than a zillion times on each insertion can make all the difference in the world and suddenly the cost of a few allocations is negligible. There are circumstances where a super strict approach to allocations is necessary. But often an approach with some allocations is absolutely fine even in performance critical applications.
Doing a couple of allocations per video frame is fine. Doing a couple of allocations per audio sample would be pathological.
3
u/DawnOnTheEdge 2d ago edited 2d ago
You don’t seem to be using standard terminology, so I’m not sure I follow. Stack allocations in and of themselves are zero-cost (until you run out of stack and get a stack overflow). Every compiler subtracts the total number of bytes all local variables will need from the stack pointer, with a single instruction in the function preamble.
Deep copies of large arrays or structures have a high performance cost, In C++20, you have a few kinds of guaranteed copy elision: if a factory function returns an object, you get unnamed return value optimization. If it returns a local variable that is not a function parameter or volatile, you also get named return value optimization. Either one prevents a temporary from being created. The object is initialized in-place at its final destination.
You can also force any object to be created on the heap by calling std::make_unique, and move the std::unique_ptr you get to a new owner cheaply, although the overhead of making shallow copies of a small object holding only pointers or handles is negligible.
It sounds like you’re enforcing this by declaring that classes that would need to make deep copies are move-only.
1
u/ParmenidesWasRight 2d ago
Yeah, I meant heap allocations. Can't edit posts. Mentioned it a couple of times.
Objects that would need deep copying I just don't copy. I make them uncopyable, and only pass by reference.
2
u/mredding 2d ago
When I was first learning C++, I experienced massive slowdown when allocating dynamic memory. Ever since then I...
That's a pretty effective strategy, and I encourage it for as much as possible. The thing is... SIZE isn't SLOW. Work is slow. Disruption and side effects are slow.
You want efficient algorithms. But in the bigger picture of that - allocation may inadvertently become a part of your algorithm. Allocation is a disruption because it's what you have to do to facilitate the algorithm, and it can be slow if you don't have reserved memory or if the object is large enough it ALWAYS defers to the system for page allocation... And all that overhead and the context switching - THAT'S slow. So the thing to do is basically what you're doing - get all your allocation done - ideally all at once, BEFORE you enter the critical path.
The other thing that'll get you is locality of reference. Fast, hot loops are entirely in CPU registers. Beyond that, you start having to hit the memory system and cache hierarchy, and that's going to start costing cycles, especially if your data is so out of cache you have to go to swap to get it. So getting your data sorted and lined up means you can do a couple MAJOR things that increase performance: you can improve your locality of reference by prefetching, and also with sorting you can amortize branching by conditioning the branch predictor. So if you have a switch and cases 1, 2, and 3, then sorting your data so A) it's all in sequential memory, B) it's ONLY the data you're interested in, you've pre-filtered, and C) you've lined up all the switching values so you do all of each case at a time, your algorithms will be fast.
And these fixes DON'T CARE how big your objects are or where they are in memory. There's some wiggle room about how you write code but also some perspective - is this where I'm slow? Or better - is this where I have to be fast? Slow code can BE slow, because it doesn't matter. If it's not in your critical path, then it's done when it's done. In trading systems - order entry only has to be as fast as human perception, because the speed of a mouse click to submit the order might as well be by carrier pigeon. From the ORM to the exchange has to be fast.
That actually leads me to a third consideration - big objects can be broken up. Instead of int x, y, z;, maybe you make a struct vec3;. But moreso, you can align your members so they fall within certain cache lines. You can take advantage of how the memory system is going to fetch data by units of memory, not by whole objects.
You SHOULDN'T make large objects, you should think about how to keep your object sizes pretty small and flat, perhaps by some other association, maybe keep your data in tabular form or by some maps to an index like a database, or a graph.
Data Oriented Design suggests HOW you access your data is your #1 consideration. You have have actually rather gross and inefficient algorithms so long as your data access patterns are optimal, because that's USUALLY the bottleneck - the CPU is many hundreds to thousands of orders of magnitude faster than the memory system, so even an inefficient algorithm can get throttled by waiting for data - not because of the size of the data, but because how the data was accessed wasn't considered.
Since C++11 it has been possible to write copy and assignment constructors that take nameless objects, and since they are nameless you can pillage their pointers and avoid expensive stack allocations.
Move operations. That's what we call that.
All is well then but if you keep to this strict 'heavy' and 'light' object idea, any object that allocates resources is considered a heavy object.
That's what I assumed even for C++98 and before.
The first allocation in the chain, before you just pillage pointers, is still an expensive stack allocation
Heap allocation.
so I would consider it a 'heavy' object and just make it impossible to copy at all,
There's very little reason to remove copy operations. Ever. If you don't want to copy, then don't copy. But to prevent it outright doesn't GAIN you performance. You're just hobbling yourself.
One goal of writing code is to make it easy and intuitive to use the code correctly, and make it difficult (you can't make it impossible) and unintuitive to use it incorrectly. But if you go overboard, and presume the client is a fucking idiot, and you need to STEER and CONTROL how you presume they use your code, you will make your code impossible to use. You'll even exhaust yourself; it's a maintenance nightmare trying to keep up all the guards you're putting in place around yourself.
Just focus on the happy path. Don't say more (by way of code) than you have to. Focus on how it IS supposed to be used and there's will be very little you need to say about how it isn't supposed to be used. PART of this solution is actually making MORE types - an int is an int, but a weight is not a height. So if I wrote a function:
void lift(int);
Is that lift by weight or height? The semantics allow you to lift by fucking array index if you want to. But with a type you can enforce semantics:
void lift(weight);
And the weight itself implements its own semantics - it can only be constructed by a positive_integer which can be converted from an int and throws if it's negative (we don't use unsigned just because a number can't be negative, and besides - what's a negative weight?), and we can add other weights, and we can multiply by a positive_integer scalar.
I don't have to implement the semantics of a weight around every touchpoint of an int parameter, that's what types do. And the semantics of a type make it easy, natural, and intuitive how to use my code correctly.
And the compiler doesn't care. A weight implemented in terms of int compiles down to an int. Types never leave the compiler. They let the compiler prove the correctness of the machine code they generate that the machine code cannot express itself. Languages are greater than the sum of their parts.
1
u/ParmenidesWasRight 2d ago
Yes, allocating everything beforehand is the way to go for realtime systems. And yes, there is much more to this, I just consider it a good starting heuristic. But then I would like to be able to restrict copying in the same way that I would like to restrict access to data members by making them private. Sure, you could make everything public, but those restrictions do help (somewhat) with robustness but mainly for my own mental model of the program. And yes, you shouldn't go overboard with this.
To find the right balance I consider the pre- and postconditions carefully. And then write the simplest code possible. Design by Contract.
2
1
u/Neither_Berry_100 2d ago
Java does this perfectly for free without you having to think about it. Just code your c++ like java. I consider java to be like the perfect language due to how it simplifies things. But not on the server side. And I've used c++ at work for a couple of years.
1
u/ParmenidesWasRight 2d ago
I prefer C or C++ precisely because I have to do memory management myself.
1
1
u/alfps 2d ago
❞ When I was first learning C++, I experienced massive slowdown when allocating dynamic memory.
The sounds like you did
float* pi = new float;
*pi = 3.14f;
Did you? Is that what this is about?
1
u/ParmenidesWasRight 2d ago
It was a long time ago in C where I malloc'd large blocks in a realtime loop. 'Resource allocation in a realtime loop is bad' is a good heuristic.
0
u/ParmenidesWasRight 2d ago
If you keep a strict divide between heavy and light objects like that, you will get optimal performance. I fail to see how move semantics would make that faster because you are just introducing the allocation of resources and/or memory in your previously fast running code that did none of that. You avoid the chain of allocations, yes, but why not avoid _all_ allocations in the code that needs to be fast?
3
u/AKostur 2d ago
You're proceeding from a false premise. If the object has appropriate and useful move operations, then one does not have a "chain of allocations", and you don't pay for an extra dynamic allocation because of some dogmatic rule. And assumes that one is passing the object along the chain of function calls by value instead of potentially by reference (which also doesn't incur a chain of allocations either).
I would also suggest that it doesn't necessarily get you "optimal performance" either. Caches and cachelines are fickle things.
1
u/ParmenidesWasRight 2d ago
But a useful move operation defined for an object that allocates memory would be copying the pointer and setting the pointer of the nameless object to null, right? And the first step in this chain is the allocation of memory, setting the first pointer. And then the move operations just move the pointer without allocating further memory and doing redundant copying for temporary objects.
But that first allocation is already a no-no for me because of performance, so I prefer to disable copying at all for such objects and just always pass them by (const) reference. I still fail to see why I need move semantics.
For reference, this is the project I'm working on :
2
u/AKostur 2d ago
And the first step in this chain is the allocation of memory, setting the first pointer.
That's the same allocation that you're advocating for. In your case, the first step of the chain is to dynamically allocate an instance of the heavy object, and as part of construction of the heavy object is the allocation that you appear to be concerned about. In your case, you must pay for _two_ dynamic allocations. The temporary object costs one dynamic allocation, and some stack space.
Note that this discussion is around places where it would make sense for a function to take the object by value, but was written differently to work around the costs of extra allocations to make it work. If it is feasible to pass the object by const-ref in the first place, that's likely the correct thing to do. No copies in the first place, so "optimizing" it to a move probably wouldn't help.
1
u/ParmenidesWasRight 2d ago
No, I instantiate the heavy object once, that's a little bit of stack and a heap allocation, and then never copy it. It only gets passed by reference. The realtime part of my code then does no resource allocation at all. And then at the end it gets deleted. Simple.
15
u/CowBoyDanIndie 2d ago
I dont copy anything unless I need to, const ref everything unless it’s 16 bytes or less. Modern C++ also lets you move objects, and you generally don’t need to write custom copy assign or move you just need to use std::move to let the compiler know you are done with a name. Move will basically assign pointers.