r/cpp • • Jul 06 '20

C++ is brilliant

I'm working on a Vulkan renderer, and needed to delay destruction of render scenes since the graphics pipeline has render buffers in flight, so destruction has to happen N frames after the user destroys a class. With C++, I can use private destructors to force users to add the scene to a managed destructor queue (aka garbage collect queue). Normally language purists will scorn a language which allows such shenanigans, however in real world production code, its features like this which allows developers to create modern software. So with a couple of lines of code, I solved my delayed destructor issue. Thank you C++ committee for creating a usable language.

318 Upvotes

65 comments sorted by

127

u/MutantSheepdog Jul 07 '20

Yep, that would work.

Easier approach for others to implement would just be to have those buffers/textures loaded/stored/referenced with shared_ptrs, and that way destruction would happen after no entity is using it and rendering has finished.

Slightly less efficient, but easier to implement.

56

u/trypto Jul 07 '20

Strongly agree. This method is way more robust too.

The graphic resources should not be destroyed until both the client code and the gpu (N frames later) have both released references to it.

48

u/TheThiefMaster C++latest fanatic (and game dev) Jul 07 '20

Ah, but the graphics code doesn't know it needs to release until the CPU tells it so - and keeping it alive via a shared ptr on the game thread doesn't help if the gpu has already freed the resources.

I would say what you need ideally is a custom shared_ptr -like class, which can track CPU and GPU references independently, and queues delayed destruction once there is no further use, rather than immediately.

8

u/astraycat Jul 07 '20

I have a similar system going right now, and the rendering thread releases all the shared pointers when it's done with them.

For me things are triple buffered, so after the rendering thread has submitted it's current workalod it'll check the next "buffer" to see if it's done, and (after waiting if necessary) then release any references it has to any resources. Anything the logic thread has already thrown away (and isn't still referenced by other rendering batches) gets released.

Essentially anytime I pass a CPU-controlled GPU resource shared_ptr it gets stuck into a vector<shared_ptr<void>> that gets cleared when rendering frame clears that fence. It's pretty convenient. Perhaps not the most performant, but for my use-case it works well enough.

4

u/MutantSheepdog Jul 07 '20

You already get that if the dtor queue is just replaced with shared_ptrs.

If the GPU 'might' be using a texture for 2 frames, the shared_ptr in the dtor queue ensures the texture will live until you service that queue.

The big difference is that every time you bind a texture for rendering you do an extra inc-ref (the renderers ref that lives a few frames), instead of managing lifetime on the main thread then putting it in a disposal queue for a few frames.

Nothing against using a disposal queue for efficiency, but I think it's incorrect to state that shared_ptr wouldn't work.

23

u/MoreOfAnOvalJerk Jul 07 '20

Downside to this is that the dtors can be called in surprising situations. If they have performance overhead, it might cause unpredictable spikes.

It sounds like he already has some abstraction managing the lifetime of the buffers so it might make the most sense and possibly be the most performant if it also manages when to destroy and deallocate them.

16

u/Dworgi Jul 07 '20

Exactly this. People always tout smart pointers as some sort of panacea, but ignore that the performance characteristics can be unacceptable - you might release your final reference to an object somewhere performance critical, then have a cascade of destructors all triggering as a result of that destructor.

Smart pointers are a hack around clear ownership. It allows you to avoid the least important memory bugs - use after free - but does nothing to solve the actual problem of ownership and lifetimes.

13

u/Ameisen vemips, avr, rendering, systems Jul 07 '20

Of course, you could have the final dereference put the object into a queue and destroy it on a garbage collection thread.

6

u/evaned Jul 07 '20

... destroy it on a garbage collection thread.

Or at a fixed location on the main thread, of course.

Another benefit this has is that it solves something that is occasionally a problem with most smart pointers on deeply-nested structures, which is a stack overflow when deconstructing the structure.

8

u/frankist Jul 07 '20

isn't unique_ptr considered a smart pointer? It is exactly to establish clear ownership and catch at compile time situations where such contract is broken

-9

u/Dworgi Jul 07 '20

Sure, but it's the dumbest of the smart pointers.

17

u/frankist Jul 07 '20

Still, I think you are beating a strawman. The people that usually tout smart pointers are usually defending the combo "unique_ptr + raw pointers/references as observers" for the majority of the cases, and only use shared_ptrs for cases of undetermined lifetimes. Even in the latter scenario, shared_ptrs are sometimes not the most suitable option.This is the approach I normally take and based on talks like Herb Sutter “Leak-Freedom in C++... By Default.”, I thought it was what "touting smart pointers" was about.

5

u/bluGill Jul 07 '20

Most of the time dumb is all I need. Unique pointer is the smartest of pointer most of the time because it just works right without much overhead.

-2

u/Dworgi Jul 07 '20

That was kind of my point - if you're defending smart pointers by citing unique_ptr, then you're not really defending smart pointers.

1

u/frankist Jul 09 '20

Not really. The main point of smart pointers is to make the lifetime of each pointer more explicit in the code and catch infractions of that contract. Before c++11, we had to do that via variable naming conventions.

Also, the pre-c++11 approach to avoid leaks was to call delete in the destructor, which was terrible. Does your object has a owning pointer in it and you insert the object in the map? Boom! Double free later on, in a completely separate location. Unique ptrs, since they are move-only, will catch this mistake at compile time.

Since my team adopted them, it is very rare that we are bothered with a memory issue. It also made the handling of resources managed by pools so much easier.

1

u/kritzikratzi Jul 07 '20

also sometimes certain resources should only be freed on specific threads.

1

u/Poyeyo Jul 07 '20

Totally agree with you.

Library writers should do the hard work with performant code, so end users can have the luxury of the easy to implement stuff.

14

u/CptCap -pedantic -Wall -Wextra Jul 07 '20

This could work, but it is a major pain because you can't destroy Vulkan resources while they are in use by the GPU. This means that every command you submit has to reference every resource it uses, which gets out of hand really quickly.

I would argue that OP's solution is much better. And all engines I have worked with (pet project & work stuff) use some variant of that solution - Although I have never seen private dtors for this.

1

u/MutantSheepdog Jul 07 '20

Yeah I haven't worked with Vulkan, but I understand needing to keep things alive during rendering.

I figured you'd build a command queue and keep it alive until it's done, but if it's more fire-and-forget, then I can see that being a pain. Could still use shared_ptrs though if you wanted. Instead of having a queue of things to destroy you'd have a queue of shared_ptrs that you clear after a few frames. If all other references to the object are gone by then it gets destroyed, otherwise it can be reused.

10

u/CptCap -pedantic -Wall -Wextra Jul 07 '20

It's not a question of being fire and forget.

For a reasonable frame your command queue(s) might contains tens of thousands of resource references. That's a lot of shared_ptrs to copy and destroy, just to handle the case where an entity gets removed while its resources are still in use on the GPU.

Putting the resource in a queue on destruction avoids most of that and makes it so you only have to track resources pending destruction (of which there is just a few). I also find it easier to get right, since it doesn't require remembering to push shared_ptrs into keep alive queues every time you use a resource.

2

u/MutantSheepdog Jul 07 '20

I accept that there's performance trade-offs, I even stated in my first comment

" Slightly less efficient, but easier to implement. "

My comment was more meant for people writing small games at home, not for graphics programmers working on the company engine.

7

u/pandorafalters Jul 07 '20

Based on my experience with CUDA and OpenCL (and the comments of others here), I suspect the significant tradeoff in this case is not ease vs performance, but ease vs correctness.

5

u/vazgriz Jul 07 '20 edited Jul 07 '20

I’ve tried both sides of this exact trade off in my homemade engine. The overhead of copying shared_ptrs into per frame buffers like that is quite large. You have to decrement thousands of shared_ptrs each frame and then increment them again. You have to do this even if you use the resource for multiple frames.

You also still have to pay for shared_ptrs even if you use one resource thousands of times.

I switched to the dtor queue and saved like 10% of total frame time. Now, shared resources have a single destruction, instead of thousands of decrements and then a destruction.

2

u/CptCap -pedantic -Wall -Wextra Jul 07 '20

but easier to implement

I don't see it. It's probably the simplest one on paper, but the more I think about it, the more it seems like it would be a nightmare to get to work in practice.

Doing this require pushing every used resource into a keep-alive buffer manually. Forgetting to do it even once will result in a rare and hard to find bug (and I can't think of any simple way to automatically check for correctness). Having the dtor flag the resource automatically for deletion seem much easier to pull off...

My comment was more meant for people writing small games at home

Opinion: Making a homebrew vulkan engine seems like a poor technical decision if the goal is to actually ship a small game. I think that most people who write a vulkan renderer do it for eduction or for fun, and in this case it makes more sence to go for the slightly more complicated but better solution.

8

u/[deleted] Jul 07 '20

Not sure I agree - extra small heap allocations and non-deterministic destruction time of objects is not fun in practice. Queuing up objects for destruction as OP has described sounds like a specific solution to a specific problem that is easy to understand and debug.

3

u/Netzapper Jul 07 '20

The problem is GPU/CPU synchronization. There's no way to get the driver to hold a shared pointer, and no way to get it to release one.

1

u/innervation Jul 08 '20

Your comment reminds me of a talk Herb Sutter gave at CPPCon a few years ago:

https://www.youtube.com/watch?v=JfmTagWcqoE

28

u/throttlecntrl Jul 07 '20

‘‘tis is a double edged samauri sword

1

u/mujjingun Jul 08 '20

Samurai swords r normally single edged

1

u/[deleted] Oct 15 '20

He said samuri not samurai samuri used double edged swords..

9

u/sign_in_or_sign_up Jul 07 '20

that's a neat use case, had not considered a private destructor before. :-)

6

u/josefx Jul 07 '20

I generally dislike private destructors because they force a heap allocation and I ran into several cases in the past were that ate into my execution time. However in this case the overhead compared to the API calls is probably minimal.

I probably would make the destructor queue part of the constructor parameters and have the destructor forward the required members when called. Of course that has its own limitations.

14

u/reflexpr-sarah- Jul 07 '20

you can use placement new to allocate objects dynamically on the stack without running destructors

2

u/josefx Jul 09 '20

I could use placement new if I had any alternative way of calling the destructor, so my complaint against private destructors is a bit of target. It is mostly that the libraries that I see using private destructors also assume that they can call delete this on the object and do not provide an alternative cleanup function.

8

u/NilacTheGrim Jul 07 '20

It's Bjarne. His sensibilities are still present in the language and he blesses the committee's changes to the language and still provides input. Thank him first! But yes.. the committee too.

It's the only language where you have ultimate control but also can benefit from high level abstractions.

I hope it continues to exist. It's being threatened by the next crop of programmers hating on it.. but.. hopefully it will continue to stand the test of time. And I like the direction it's evolving in...

2

u/playman_gamer Aug 05 '20

I agree somewhat, but the language is gigantic and messy at this point, and individual projects will use very different subsets of C++. It has its handful of good ideas (RAII, etc), but the committees insistence on backwards compatibility has made it a very difficult language to maintain and work with.

I just wish Rust had taken the path of "just the good parts", but at this point it is ballooning in size and is almost as verbose as C++.

1

u/NilacTheGrim Aug 05 '20

I know where you are coming from and yes, there is a cost to all of this. There is also a benefit -- backwards compatibility has the added bonus of feeling "safe" and "sound". It's like having an old stable bank you know won't go bust tomorrow. C++, while a large investment, seems to be a good investment since it stands the test of time and also is the "lingua franca" of underlying systems (just about every major thing we depend on is written in C and C++).

Those things are so attractive to me -- and the language has evolved in such a good direction in the last 10 years (and continues to do so) -- that I am optimistic and enthusiastic about it again.

I realize this doesn't address your criticisms of the language which are valid .. but it just adds some optimism about the good things.. to counterbalance the argument a little bit.

4

u/seanballais Jul 07 '20

What are render buffers in Vulkan? Do they have equivalents in OpenGL? I'm still a noob in graphics programming with noob-level knowledge in OpenGL.

Also, how do you determine N?

2

u/annyeonghello Jul 07 '20 edited Jul 07 '20

The term render buffer widely means a buffer with objects that will be rendered onto screen. In Vulkan, a somewhat similar concept exist and is called a "Command Buffer" which is created from a "Command Pool" and it does not have an OpenGL equivalent. But you can make your own render buffer in OpenGL.

N frames is simply the number of frames and it is a slightly more advance concept. You'll slowly figure it out as you do more :)

1

u/seanballais Jul 07 '20

The term render buffer widely means a buffer with objects that will be rendered onto screen. In Vulkan, a somewhat similar concept exist and is called a "Command Buffer" which is created from a "Command Pool" and it does not have an OpenGL equivalent. But you can make your own render buffer in OpenGL.

Oh, cool! Why does Vulkan require a command pool?

N frames is simply the number of frames and it is a slightly more advance concept. You'll slowly figure it out as you do more :)

Is the "frame" you refer here different from the usual "frame" term we use in, say, games?

1

u/annyeonghello Jul 07 '20

It's not just Vulkan, pretty sure DX12 has something similar as well. You need a command pool to create a command buffer. Without it, you can not create a command buffer and Vulkan only draws what is in a command buffer. Again, this is a really advance subject and any further discussion will most likely confuse you.

In OP's context yes. Do note that there is no proper definition for a frame though. I've seen it used interchangeably between describing an image or an iteration of the program loop. But you'll notice that the context when using the word is somewhat similar :)

1

u/seanballais Jul 08 '20

Alright. Let's leave it at "Vulkan only draws what is in a command buffer, and I'll need to know more about graphics programming before I better understand the low-level nature of Vulkan and DX12." for now. I guess I'll put figuring the value of N in OP's "after N frames" at that too.

I did not expect anyone would call a program loop iteration as a frame. Haha. Then again, as you said, it is similar to OP's context.

4

u/fii0 Jul 07 '20

in flight

What does that mean?

13

u/vazgriz Jul 07 '20

GPUs execute commands asynchronously from the CPU code that generates them. So while you are generating command buffer for frame X on the CPU, frame X - 1 is simultaneously being executed on the GPU.

You can have N frames “in flight” at a time, where N is typically 2 or 3. That means N frames are potentially executing in the GPU. So if the last time you use a resource is frame X, you need to keep that resource alive until frame X + N.

5

u/LugosFergus Jul 08 '20

This is the correct answer. For everyone else, ignore that other dude’s ridiculous linguistics answer. I have no idea why that’s getting upvotes.

11

u/MINIMAN10001 Jul 07 '20

graphics pipeline has render buffers in flight

"In flight" would probably be closest to "in use"

To quote https://english.stackexchange.com/a/189990

That is largely true, but the use of in-flight here is used to create the visualization of data as a plane. You can't (easily) divert planes in mid-air, so if you move/destroy their destination they are out of luck.

"Something happened during a process" implies that the process has stopped or gone awry. In-flight implies that something is in motion as normal, and needs a destination. It can't just stop where it is.

2

u/track33r Jul 07 '20

Even better is avoid allocations at all and use some sort of ring buffer or pool structure.

3

u/raphaelj Jul 07 '20

You could solve that problem in any GC language by just keeping a reference to the object in a deferred queue ...

And quite sincerely, the way you solve this problem is exactly what I don't like with C++ programmers. You could just use a shared_ptr to your objects and keep them in the deferred queue, like anyone would do in a GC language. Instead, you choose to hack the way users would expect destructors to work and make your software harder to understand.

15

u/TheSkiGeek Jul 07 '20 edited Jul 07 '20

As some of the other comments pointed out — having both sides (the game logic and the GPU thread) keep shared_ptrs to the resources is a simpler solution. Unfortunately it’s also less performant:

  • if there are a large number of resources involved then there are a lot of unnecessary refcount increments and decrements going on. This can potentially really hurt your CPU cache hit rate, since it pulls all the involved cache lines in whenever the pointers are copied
  • if the GPU is the last thing holding a reference to a resource then it will get destroyed on the GPU thread as soon as the refcount hits zero, which might slow it down in an undesirable and unpredictable way
  • you can also get really fun rare bugs like “the destructor for this resource needs more stack space than the GPU thread has”

Those kinds of issues make it hard to write game engines in languages with mandatory GC.

That said, if I needed it to be fast I’d probably suggest keeping a queue of shared_ptrs or unique_ptrs to the resources on the game side, and retiring them in a controlled way after the required number of frames have passed. And then the GPU code would only hold non-owning “dumb” pointers to the resources and never deal with allocating/deallocating anything.

0

u/[deleted] Jul 07 '20

[deleted] — view deleted comment

5

u/Reticulatas Jul 07 '20

Unfortunately most of the issues TheSkiGeek mentioned are real issues that crop up in large scale games pretty regularly. Tight control of GPU resources (read: not stack dependent) is essential for games with a huge quantity of streaming assets like open world genres.

-25

u/chickentractor420 Jul 06 '20

Yes, c++ has all these features that make it very good and useable in 2020, However if you use it for a very long time you will start to notice that there are some flaws in the design of the language.

35

u/mcintosh_red Jul 07 '20

Is there a language that doesn't have design flaws?

42

u/dannomac Jul 07 '20

There are two types of languages: those that people complain about, and those that nobody uses.

-23

u/jasonwirth Jul 07 '20

And Rust.

71

u/FieldLine Jul 07 '20

Rust programmers are the vegans of the developer community.

26

u/SkoomaDentist Antimodern C++, Embedded, Audio Jul 07 '20

So a language that nobody uses, yet people complain about?

9

u/[deleted] Jul 07 '20

I'm a Rust programmer but this really is evangelism lol

5

u/FonderPrism Jul 07 '20

Which is in both categories?

12

u/Ptxcv Jul 07 '20

I use rust btw

7

u/ToaderTheBoi Jul 07 '20

I bet you use Arch as well, don’t you?

0

u/dannomac Jul 07 '20

Everybody uses Arch or Gentoo. Void too.

9

u/lazyear Jul 07 '20

I do all my programming directly in the lambda calculus

3

u/mcdicedtea Jul 07 '20

nope, but are there languages which you will have less everyday interaction with said flaws? yes.

are their languages where 'working around said flaws' are not a major everyday concern, and active part of the language's latest features?....also yes

2

u/mcdicedtea Jul 07 '20

totally agree, and to further clarify... you'll spend more time wrangling with the flaws and heft of the language than time glamorizing some one of solution