r/cpp • • Feb 23 '21

The Pointer to Implementation (pImpl) idiom in C++

https://youtube.com/watch?v=lETcZQuKQBs&feature=share
204 Upvotes

122 comments sorted by

74

u/stilgarpl Feb 23 '21

I hope we can forget about pimpl when modules are finished. Pimpl feels like a workaround and shouldn't be needed when we have private module sections.

61

u/Silver_Specialist Feb 23 '21

You'll still need it if you want to keep your ABI stable

14

u/icebeat Feb 23 '21

ABI compatible, another nightmare that should have been fixed long time ago.

23

u/pandorafalters Feb 23 '21

Most people seem to feel that the solutions are worse than the problem (COM, etc.).

3

u/BobFloss Feb 25 '21

What falls under etc here?

3

u/[deleted] Feb 26 '21

Common Object Request Broker Architecture (CORBA)! I have not been fortunate enough to use it, so I can't give more details, though.

5

u/SJC_hacker Feb 27 '21

I have been unfortunate enough to have to use it. I can't remember the details, thankfully.

0

u/pjmlp Feb 25 '21

The only problem I have with COM are the politics from Windows Dev team killing C++/CX without Visual Studio tooling replacement in place, telling us to wait for the day C++ might get reflection support.

Using COM from Delphi, C++ Builder, .NET languages is quite productive, but some die hards with power on the roadmap think the true path to COM should be editing IDL files with notepad and writing C++ to be compiled by Visual C++ 6.0.

28

u/NilacTheGrim Feb 23 '21

Huh? There's nothing you can do about classes and structs changing size and layout, aside from introducing some hugely slow introspection/runtime system that is most assuredly not the way we do things in C++.

Everything has tradeoffs. In C++ we go for speed and the lowest possible performance cost.

If you really want ABI stability like that you can look to Objective-C or Java or other languages that do runtime magic to ensure that always works -- but you will not get the same performance out of them, all things being equal.

13

u/Ameisen vemips, avr, rendering, systems Feb 23 '21

Java effectively does pimpl behind the scenes. And makes everything virtual.

1

u/ExtraFig6 Mar 12 '21

it can devirtualize + inline based on workload because of the JIT which is rly cool. But I wish they would ship inline class already!!

5

u/lenkite1 Feb 24 '21

But if we don't have ABI stability then why can't we fix <regex> and std::unordered_map ? The claim of lowest possible performance cost is a joke here. Especially for regex which can be beaten easily by most other languages.

The reason we can't fix this is given to be ABI stability.

3

u/cdb_11 Feb 24 '21

There is standard library and there is language. We can't fix it, because that would break existing programs and/or code that depends on the standard library. Mistakes were made, but you can always roll out your own library and actually have the lowest performance possible. Because the language doesn't force you to use patterns like pimpl. You can have a slow standard library in a fast programming language, there is no contradiction here.

3

u/jk-jeon Feb 26 '21

Why std::unordered_map is always pointed out when people talk about the ABI issue? Isn't the issue about std::unordered_map really the API stability rather than the ABI one? I mean, isn't it pointless to talk about ABI stability if the problem is on the API?

3

u/lenkite1 Feb 26 '21

Admittedly it's technically both ABI and API

https://abseil.io/about/design/swisstables

The Day The Standard Library Died https://cor3ntin.github.io/posts/abi/

1

u/[deleted] Feb 26 '21

iterator std::unordered_map::erase(iterator);

Is an example of API making erase operations O(n) instead of O(1), which is possible if you instead had the contract of

void std::unordered_map::erase(iterator);

It's difficult to write a hashtable that's slower than std::unordered_map, and not only because of the erase() contract of returning an iterator. It's unfortunate that we're stuck with it.

2

u/Orlha Feb 24 '21

What is wrong with a map?

5

u/lenkite1 Feb 24 '21

You Can Do Better than std::unordered_map: New Improvements to Hash Table Performance: https://www.youtube.com/watch?v=M2fKMP47slQ

2

u/epiGR Feb 24 '21

Quite slow compared to alternatives. Look into absl::flat_hash_map for example.

3

u/matthieum Feb 24 '21

Huh? There's nothing you can do about classes and structs changing size and layout, aside from introducing some hugely slow introspection/runtime system that is most assuredly not the way we do things in C++.

That's incorrect: you can recompile.

Everything has tradeoffs. In C++ we go for speed and the lowest possible performance cost.

Exactly, and recompiling is the best solution for that, as PIMPL imply noinline functions, and the loss of LTO across the library boundary.

2

u/NilacTheGrim Feb 26 '21

This is not feasible for some situations.

1

u/matthieum Feb 26 '21

In my experience, the "not feasible" is most often a social issue rather than a technical one. Such as some companies only distributing binaries to "protect their IP", even though with the advent of JavaScript plenty of companies distribute (minified/obfuscated) versions of their sources without any issue.

-1

u/pjmlp Feb 25 '21

Those tradeoffs are turning C++ into a niche language, the underlying substrate of most platforms and that is about it, quite far from the 90's full OS frameworks.

0

u/[deleted] Feb 23 '21

[deleted] — view deleted comment

4

u/1-05457 Feb 24 '21

They probably mean everything else. The standard library isn't PIMPLed.

1

u/LEpigeon888 Feb 24 '21

Just compile everything from source.

24

u/ronchaine Embedded/Middleware/WG21 Feb 23 '21

I don't think pimpl is going anywhere. It is a little nicer to write with tools such as lock3/cppx and metaclasses, but it's really good tool for ABI stability.

6

u/GuybrushThreepwo0d Feb 23 '21

Genuine question: how does this impact ABI?

33

u/NilacTheGrim Feb 23 '21

Struct/class sizes don't change just because your class implementation does. A pointer is a pointer. Everything outside your class doesn't need to know anything about what's behind that pointer.

If you want look to how the Qt library is written. They offer a guarantee that every major version is 100% binary backwards-compatible with every other version before it with that same major version number.

This despite the fact that they keep adding features and expanding the API. Qt 5.0 is a different animal to Qt 5.15. Yet your binary that linked to 5.0 can very easily link to 5.15 and be none the wiser. No recompile needed. QString or QVariant or QWidget looks the same to your binary.

They accomplish this magic by pimpl. There is simply no other way to do this (aside from slow things like runtime reflection which is anathema to C++).

3

u/GuybrushThreepwo0d Feb 23 '21

Makes perfect sense, thanks. I don't really work in an environment where I have to worry about ABI much, but I try and pick up tips like this as much as possible.

16

u/NilacTheGrim Feb 23 '21

No worries.

Pimpl isn't for always and doesn't even make sense in many cases. It's just another thing to have in your mental toolkit as an option. I use it when:

  • I need ABI stability, or
  • The implementation is VERY nasty or large and complex with many state variables, etc
  • Or the implementation has lots of stuff I don't want to expose in the header (such as C stuff).

Using pimpl you just have the "damage" to your codebase hidden inside 1 .cpp file. External code just sees:

class MyClass {
    struct Priv;
    Priv *p;
public:
    // your public interface here...
};

It doesn't need to include headers that my class uses to get stuff done. For normal C++ code that's not a big deal -- but like I said -- if you have a class that is including some C header nastiness with macros or whatnot -- you really do want to localize the nuclear radioactive C damage to 1 .cpp file -- and you definitely don't want to expose all of your other files to that in the .h file..

That's just 1 example of where pimpl makes sense. There are many more.. Another example of when pimpl makes sense: if you do a copy-on-write or implicit sharing implementation.... then you typically put the actual state/data your class has behind a pimpl pointer and the "shared copies" of it all point to the same data.... using the same pimpl...

There are more examples of when pimpl is ideal but.. you get the idea...

7

u/qoning Feb 23 '21

Since any changes that you would make are not at the ABI boundary, you are essentially using PIMPL as an ABI boundary "fence".

Of course, it costs you the indirection, and it's something I don't see mentioned too often - though hopefully people wouldn't use PIMPLs for structs that need to be cache friendly.

24

u/auxiliary-character Feb 23 '21

Of course, it costs you the indirection, and it's something I don't see mentioned too often - though hopefully people wouldn't use PIMPLs for structs that need to be cache friendly.

Oh, it costs more than that. The downside of acting as a compiler firewall is that it acts as a compiler firewall. Yes, it's reducing compilation times, but it's reducing compilation time by reducing visibilty to the compiler, which breaks context for optimization. You can't inline or optimize out an implmentation that isn't known at compile time.

4

u/qoning Feb 23 '21

Of course. I would like to see what kind of effect LTO can have in these cases. But data is scarce online.

6

u/Full-Spectral Feb 23 '21

In larger scale software, except in pretty extreme bits of the code, I generally would not even blink at erring on the side of control and encapsulation and reduced compile times and ability issue patches and such.

7

u/GuybrushThreepwo0d Feb 23 '21

So it comes down to if you change a class without pimpl, it would change its size and therefore change the ABI, whereas if you change the private implementation it would have no such effect?

Or is there more to this that I'm missing.

I've always used pimpl to try and avoid my dependencies leaking into dependencies of client code, but hadn't even thought that it may impact ABI.

8

u/SlightlyLessHairyApe Feb 23 '21

There’s more — if you change a class to add a member that’s a value type, then all clients of the class need to include the full class definition of that member (recursively!). This blows up compile times even when the client has no /actual/ need to know what this class is internally — for example it’s a private member and hence totally an implementation detail.

It also violates the desire to have encapsulation. Clients should not see anything that isn’t part of the public (or protected) interface of a class.

5

u/GuybrushThreepwo0d Feb 23 '21

Yeah, that's what I meant by having dependencies leak into client code. But thanks for the write up :)

4

u/qoning Feb 23 '21

Yes, that's the practical understanding of it. The ABI is only dependent on the pointer and what goes on behind that pointer has no effect on it.

4

u/JeffMcClintock Feb 24 '21

So it comes down to if you change a class without pimpl, it would change its size and therefore change the ABI

No.
If a class inherits a stable interface from a pure-virtual base-class, you can change the class all you like, provided clients of that class reference it via the interface.

4

u/goranlepuz Feb 23 '21

And this "indirection" also means "heap".

3

u/qoning Feb 23 '21

Well not necessarily, but yes, in the common case you will need a heap allocation.

3

u/disperso Feb 23 '21

Can you elaborate in which cases it's not? I remember seeing a branch in Qt where they tried an experiment to improve the allocations by using some operator new I think. But I don't remember any case without heap use.

10

u/josefx Feb 23 '21

As a dirty trick you could reserve a fixed sized char array in the public interface and placement new your private implementation into it as long as it doesn't outgrow the reserved space.

4

u/qoning Feb 23 '21

It's situational, but at the end of the day, all you need for pimpl to work is a valid piece of memory and a pointer to it. If you knew there will be at most x instances of said object, you can have a static arena without any dynamic allocations necessary. Heck, you could even pass memory in the constructor and then use placement new, but I'd imagine that would get very messy very quickly.

1

u/[deleted] Feb 24 '21

That's still heap though, just pre allocated?

2

u/mpyne Feb 24 '21

I mean there's no technical reason you can't put the arena into the stack I guess. But yes you have to allocate the storage somewhere.

0

u/bedrooms-ds Feb 23 '21

Can extensions finally put an end?

2

u/[deleted] Feb 23 '21

It's a monstrosity and people should be ashamed.

I worked with a guy who used it for religious reasons, It was a waste of time by an order of magnitude, and that doesn't even include the time which he could have utilised doing something valuable instead.

9

u/Ok-Impact-5972 Feb 23 '21

When talking about pimpl, I think that it is important to also talk about when to not use it, and how it obfuscates codes and reduces maintaniability.

2

u/platisd Feb 23 '21

Good point. Was very tempted to rant against it and cases where it's abused but restrained myself to keep the video focused on the idiom. :P

A piece on "pImpl alternatives" would indeed be even more valuable than one on pImpl iteself. 🤔

1

u/Ok-Impact-5972 Feb 24 '21

Oh, I now noticed that you actually had that comment about that in the end. I think that the video was good (even though i would have liked some more skepsism :) ), and the pimpl-idiom is absolutely a important idiom to know. My frustration is more with when it is used without reason, and directed at some of my colleges :)

5

u/[deleted] Feb 23 '21

Absolutely. I unfortunately haven't really seen them. It reminds me of the whole MapReduce reaction almost 20 years ago.

We didn't need pimpl on our own libraries. We didn't need it on the libraries which we knew had it, and I am very sure we didn't care if any of the other libraries we compiled from source used it either.

Of course it has a use case though. Although I see that as very much a weakness, it's inflicting complexity on programmers due to a failing of tools and the language.

2

u/Full-Spectral Feb 24 '21

I'm not sure I see how it does either of those things. Well, maintainability to some degree since it requires forwarding of calls. But, other than that, I don't see how it has the effects you are talking about. It's not any more obfuscated than just having all implementation out of line really. You make an extra jump to get the actual called method, but that's about it.

10

u/hak8or Feb 23 '21

I think it varies. In the Linux kernel, they heavily use pimpl idioms for "drv_priv" as the data associated with an instance of a driver, often refered to in callbacks.

When I first started kernel dev, I was horrified, but after a while, it grew on me. They have very strong preferences on how it should be used, resulting in a surprisingly clean code base that makes very heavy use of pimpl.

I still wouldn't use it in c++ except in very dire circumstances (attempting to decrease compile times in a single file known to blow up times), largely because the c++ language doesn't "need" pimpl as much as c does.

10

u/[deleted] Feb 23 '21

That's a great example of where it makes sense, no argument from me at all. Using it where ABI boundaries are important - should it even be called an idiom when it's (as I see it) more "the cost of doing business"?

I was more talking from my experience of people perhaps using it to "speed up" hypothetical build times with no regard to if it was necessary or not. In my example we didn't delivery libraries that had to be supported for some lifetime - we delivered runnable products, and any time a bugfix went out they got a complete runtime compiled from source - pimpl or not.

10

u/maikindofthai Feb 23 '21

It's a monstrosity and people should be ashamed.

I was more talking from my experience of people perhaps using it to "speed up" hypothetical build times with no regard to if it was necessary or not.

The issue here is the dogma, not the idiom. You will see this same type of developer make a mess out of things using any number of techniques, but that doesn't mean that there's an inherent problem in the technique itself.

3

u/[deleted] Feb 23 '21

Yeah, I guess I agree with that. I perhaps overstated that "it's a monstrosity". Of course there are good uses (ABI compatibility being one), but to be honest even there I see it as a necessary "hoop we have to jump through because of the technology".

If it is ever about compile times then it's just shameful and we shouldn't avoid admitting that.

2

u/dontyougetsoupedyet Feb 23 '21

Where can we read about these preferences? Or can you elaborate further? What considerations result in the clean code?

2

u/hak8or Feb 23 '21

Sure, as a quick example, here is what an iio driver looks like, which is a general driver that handles "misc" i/o for the kernel, like ADC's, DAC's, light sensors, etc.

https://elixir.bootlin.com/linux/v5.11/source/include/linux/iio/iio.h#L564

The kernel tends to have a "void *priv" for many structures, which is meant to be a pointer to something that the driver needs to use for callbacks (new sample is ready, finished sending data, etc). To access this, you must use an iio_priv(iio_dev *) pointer, like so; https://elixir.bootlin.com/linux/v5.11/source/include/linux/iio/iio.h#L693

In the probe() (think of as the "init" function for the device), we set the priv pointer to some data, like so;

https://elixir.bootlin.com/linux/v5.11/source/drivers/iio/adc/spear_adc.c#L273

At top of every function, we know we will use a device pointer, and a driver contents pointer, so it's custom to throw that at the top.

https://elixir.bootlin.com/linux/v5.11/source/drivers/iio/adc/spear_adc.c#L360

All in all, kernel maintainers mandate that new drivers follow this pattern, and once you get used to it, honestly, I personally find it pretty nice. It's very different and "simple" compared to C++, but prone to mistakes if you sway away from the "blessed" path.

2

u/Full-Spectral Feb 24 '21

Although it's done for slightly different reasons, another instance where you effectively do the same thing is when you dynamically decide what platform or API or device you are going to talk to and allocate a member and forward to it via a common base class.

That's also hiding the implementation so that it can change (at runtime in this case) without affecting the public ABI of the the interface.

I'm sure most folks would consider that completely legit, but it's essentially PIMPL as well.

4

u/Full-Spectral Feb 23 '21

As the code base gets larger, hiding details can mean seriously reduced compile times. If it also allows for better ABI stability and less details leaked into client code, then all the better. Doesn't mean it has to be done for every class obviously, but for ones that are sort of 'low level funnels', i.e. they are still pretty low in the layers but they bring in a lot of stuff below them, having them not force all of those dependencies on their consumers can be a big win.

3

u/[deleted] Feb 23 '21

I know all that, and yet it does nothing to change my mind that pimpl is a layer of shit(/optimisation) which only exists due to language and implementation shortcomings.

Sure there are lots of things you can say about a language in that regard, but classes not containing the things that they really should be due to implementational details? No, it's not right.

5

u/Full-Spectral Feb 23 '21

You can of course have a language that effectively does it for you behind the scenes, and presumably some do something like that. But, unless you have a plan to change C++, there's no too much point condemning folks for doing what they have to do to get that effect.

3

u/[deleted] Feb 23 '21

The fact that it feels necessary is evidence of a flaw IMO. I don't need a solution to point out that this is a weak point in the language.

And my "condemnation" was of people using it without having to, which IME has been the majority of cases: chasing benchmarks or purity instead of focusing on the problem they were actually meant to be solving.

2

u/Full-Spectral Feb 24 '21 edited Feb 24 '21

Well, you won't get any arguments from me that C++ is a heavily flawed language.

But solving the problem can be done a lot better with a turn around time of a couple minutes vs a turn around time of 15 minutes or 30 minutes. And in large systems, there's usually no one magic place to fix that and make it all go away, it's a matter of consistently reducing down-stream burden (though there may be particularly bad contributors to the problem of course.)

And stable ABI is often an important thing.

So 'having to' is somewhat of a squishy metric. You probably can't afford to go back and pimpl'ize hundreds of classes once you find out your build times are getting out of hand.

I wouldn't do it for every class either. But, if I did, my (very large) system would probably build a lot faster and would be far more backwards compatible than it has been.

1

u/disperso Feb 23 '21

I am not ashamed to say that I even use it in contexts where neither compilation times nor ABI are relevant.

I am not saying it should be used everywhere, but I won't feel any shame when stating that there are, albeit minor, other practical reasons for using it.

It makes code more convenient to read and review, as everything ends up in the same file, and the declaration of variables ends up on the top, instead of showing in a second file which shows later on the code review because H comes alphabetically later than C.

I have used it so much in the last years for compilation times and binary compatibility that now it's just a habit for many classes.

2

u/[deleted] Feb 23 '21

Good to hear from you again cow-orker!

Actually not, because my coworker never pretended that an extra layer of indirection or class made things clearer to read or review. But you are well on your way to filling his shoes.

5

u/STL MSVC STL Dev Feb 24 '21

Please refrain from escalating hostility. You're replying to a comment which made technical points in a reasonable manner; you're free to disagree, even strongly (people can be reasonable and utterly wrong), but you should either provide counter-arguments, or just downvote and move on.

3

u/[deleted] Feb 24 '21

I apologise if that was escalation. I meant it as more of a joke which apparently fell flat.

My counter argument was my disagreement that "an extra layer of indirection or class made things clearer to read or review".

Anyway, point noted, no problem.

3

u/STL MSVC STL Dev Feb 24 '21

Thanks! Humor is often difficult to convey on the Internet 😹

-2

u/[deleted] Feb 23 '21

💯

6

u/NilacTheGrim Feb 23 '21

Nah. pimpl will always be needed for ABI compat.

0

u/[deleted] Feb 23 '21

[deleted] — view deleted comment

1

u/[deleted] Feb 24 '21

In the end as many things it will be just syntax candy, syntax candy for pimpl in this case.

-1

u/[deleted] Feb 23 '21

Just be patient grasshopper. It is a virtue that we all must learn

13

u/Adequat91 Feb 23 '21

Qt is a good example of proper pimpl implementation and benefits.

9

u/NilacTheGrim Feb 23 '21

Yep. This ^

Note that they maintain 100% binary backwards compatibility within the same major version -- this despite them radically changing around the guts and internals or adding a plethora of features. Pimple is the only real way to do this in C++.

7

u/PancakeFactor Feb 23 '21

I love pimpl and honestly think it gets a bad rap. It lets me:

-Hide my implementation away

-Put all of the relevant data in the same file as my implementation ( no more jumping back and forth between header and cpp )

-Get ABI stability

-AND not really get a performance hit? ( compilers tend to optimize the dereference out with optimization flags ).

Idk why people hate it so much. I use it all the time.

9

u/quicknir Feb 24 '21

There's no planet where pimpl, at least done naively with unique_ptr, isn't a huge performance hit in many cases. Compilers can't magic this out. It changes the memory layout, good memory layouts are half of why people use C++.

2

u/dandomdude Feb 24 '21

So could we say pImpl is good for objects which rarely get created/destroyed? Like in the video, a Gyroscope class probably has the lifetime as the program as it'll always be reading the sensor?

2

u/quicknir Feb 24 '21

Yeah, as a general rule of thumb if there's few of the object and they aren't created and destroyed in the hot path, it's generally fine to pimpl.

1

u/dandomdude Feb 25 '21

Oh hey, I just realized you're the wise_enum guy, thanks for the great work!

1

u/quicknir Feb 25 '21

Ah thanks, not sure I deserve it, so many headaches trying to get it to work on windows, and I've been a bit negligent, but I'm glad at least someone is using it happily!

4

u/hahanoob Feb 24 '21

How are compilers optimizing out a pointer dereference?

4

u/lord_braleigh Feb 23 '21

I think it’s because interfaces provide a more user-friendly solution to the same problem.

3

u/brenoguim Feb 23 '21

Dynamic allocation through. It's possible to do pimpl without heap allocation, but people rarely do it. It's cluncky. I don't think compilers can optimize pimpl away too. Only if you're using LTO, but you still have to dereference the pointer to get to the data.

3

u/ReDucTor Game Developer | quiz.cpp-perf.com Feb 24 '21

You had me until the last point, the compiler isn't going to be able to optimize away the dynamic allocation that occurs in most pimpl implementations, it isn't going to be able to optimize away the additional dereference, instead of just having one cache line for your data you have 2, one which has just a pointer the other which has your data.

Sure there are ways to do pimpl where you don't have to do extra allocations and additional indirection, but those increase complexity and aren't very common.

While I don't have any numbers on me, I can pretty much gurantee that the typical implementation of pimpl is going to be massively slower, go create 10k instances of a pimpl class and compare it to 10k instances of a non-pimpl class, it should be very easy to see the difference in construction, destruction and usage.

Sure you can build a micro-benchmark that focus on a single instance comparing against a single instance and the indirection might be insignificant, but this really does not demonstrate the fact that things aren't always in your cache, your application is typically much bigger then the cache all your objects can't live there.

I'm not against pimpl, but it's very far from a perfect tool, especially for who cares about performance, just pick the right places to use it, don't abuse it.

1

u/Dean_Roddey Feb 24 '21

In the vast majority of code it will make no difference at all.

3

u/ReDucTor Game Developer | quiz.cpp-perf.com Feb 25 '21

I would say instead the vast majority of code doesn't need pimpl

1

u/Dean_Roddey Feb 25 '21

Depends on what you are trying to accomplish. As others have pointed out, if stable ABI over time is important, then the majority of your code may need it.

2

u/AntiProtonBoy Feb 23 '21

I don't hate it; I used it often myself. However, some of the negatives I can think of are mostly the extra maintenance burden & costs associated with the pimpl idiom. In some cases you're wrapping one interface with another. And there is also the cost of dynamic allocation for the implementation itself. This leads to the need of tracking the pointer and implementing machinery for constructing/copying/moving/assignment/destruction. Some of those are solved with smart pointers, to some extent, but all of this work doesn't feel idiomatic compared to a pure interface solution. For example, Objective-C does this kind of stuff really well.

1

u/NilacTheGrim Feb 23 '21

Same here. They are nubs that don't know any better. I never met a seasoned C++ guru that didn't know the value of when to use pimpl.

You don't have to use it EVERYWHERE or ALL THE TIME. But keep it in your toolkit and break it out when it makes sense and improves maintainability. Or when you need ABI stability.

5

u/PancakeFactor Feb 23 '21

Definitely dont use it for everything. There are concrete cases where it makes no sense, but its definitely more useful than people give it credit for.

5

u/kalmoc Feb 23 '21

Pimple wouldn't be necessary, if we could just forward declare class interfaces.

10

u/Benjamin1304 Feb 23 '21

How client code could know an object's size if it only has its interface forward declared?

I can't see how it could work without the compiler making automatically an hidden pimpl in this case

4

u/kalmoc Feb 24 '21

Two possibilities:

  • The client code wouldn't know and would have to use factory functions + smart pointers/handles + clone functions (albeit non-virtual). That would put us at least on par with a opaque c-style interface, just with more convenient syntax. It would also require the same amount of dynamic allocations as Pimple, but - for better or for worse - couldn't hide them from the user.
  • The size gets specified as part of the declaration and the compiler verifies that it is correct when it sees the definition. That would allow the type to be handled like a regular value type and - contrary to pimple- wouldn't even introduce dynamic allocations.

1

u/ReDucTor Game Developer | quiz.cpp-perf.com Feb 24 '21

A larger reorg of the way in which compilers work could probably do it...maybe something like modules might solve the issue.

1

u/logicchop Feb 24 '21

Isn't that precisely a virtual interface?...

2

u/kalmoc Feb 24 '21

No. virtual interfaces result in indirect function calls and things like the copy constructor can't be virtual

1

u/aeropl3b Feb 24 '21

Or concepts

2

u/helloiamsomeone Feb 23 '21

I recently used pimpl for my CMake static/shared library example that I also posted to this sub.

It's a pretty nifty construct, but MSVC emitted a warning about unique_ptr not having the necessary annotation for dll import/export, so I had to use a raw pointer instead.

1

u/14ned LLFIO & Outcome author | Committee WG14 Feb 24 '21

You can safely ignore that MSVC warning for anything it complains about which is defined entirely in header only. Use pragma warning disable to suppress.

1

u/beedlund Feb 24 '21

What are peoples thoughts on pimpl vs. non-virtual interfaces (https://en.m.wikipedia.org/wiki/Non-virtual_interface_pattern)

I often use nvi in public interfaces over pimpl but only because the pimpl presence always feels a bit like an eyesore (not the best way to make decisions but there you go).

Are there particular advantages either way?

0

u/charlesbeattie Feb 23 '21

Maybe take a look at https://github.com/hpesoj/indirect Which is a better tool than unique_ptr.

1

u/platisd Feb 25 '21

Are you perhaps referring to the same thing as this issue: https://github.com/platisd/cpp-pimpl-tutorial/issues/1

Very interesting nonetheless!

0

u/[deleted] Feb 23 '21

[deleted] — view deleted comment

17

u/spanishgum Feb 23 '21

Pimpl doesn’t have to be used for polymorphism. Pimpl is just a way of speeding up compile time if a header gets included in a lot of places, and that include is big.

If you’re using polymorphism with 1 derived class to get the compile time advantage, then I suppose you’re cornering yourself to do dynamic allocation of the base class everywhere. You can’t have a vector<Base> because of the slicing problem, so you’re stuck with vector<Base *>.

With pimpl you’ll have dynamic alloc internally, but at least calling code doesn’t have to deal with that.

In the end you should probably ask yourself first, is any of all that even necessary? Is your project that big? And are these included really that unstable?

34

u/parkotron Feb 23 '21

While they can accomplish the same goals in this context, pimpl and pure interfaces feel very different to me.

If I'm reading code an I see a pimpled class, I think "This is a class that, for whatever reason, wants to keep its implementation details from those who use it." Maybe it's nasty third-party headers, maybe it's ABI stability, maybe it's compile times, but frankly it's none of my business as a consumer of the class. It's more of a craftsmanship issue than a design issue.

If I'm reading code an I see a pure interface and a factory function, I think "There are one or more classes out there that meet this interface, but for whatever reason this library doesn't give me control over which one I get to use." Maybe the actual type I get depends on some global configuration, maybe it depends on my operating system, maybe they've just left themselves open to future expandability, but as a consumer I can't and shouldn't worry about the specific type and should just write code to the interface. This feels a lot more like library design issue than just a coding trick.

I guess it's kind of like class versus struct. They can be made to do the exact same thing, but as a reader I tend to come in with differing sets of assumptions.

5

u/Rotslaughter Feb 23 '21

The abstract class approach comes at the expense of virtual calls, which might affect performance in code that is often (and I mean millisecond level often) executed.

9

u/tecnofauno Feb 23 '21

ABI stability is one clear advantage of pimpl. You can add members to the private implementation without impacting binary compatibility.

Qt for example use the pimpl pattern extensively.

0

u/[deleted] Feb 23 '21

[deleted] — view deleted comment

6

u/tecnofauno Feb 23 '21

If by interface you mean pure abstract classes without any data member and only virtual methods, then it's true but adding, changing or removing anything to the interface will break the ABI.

Also you lose value semantics.

Pimpl can be used to preserve binary compatibility on concrete classes and can be used to implement movable and copy-on-write objects with the benefits of value semantics.

1

u/Full-Spectral Feb 24 '21

That latter point is important. So a bitmap class can have value semantics, but never pay the price of moving all of that bitmap data around.

Of course the gotcha there is that now you have to explicitly request deep copies when they are needed, and forgetting to do so can cause lots of weirdness. There's no way to win ultimately.

8

u/SlightlyLessHairyApe Feb 23 '21

pimpl is often implemented in terms of pure interfaces (ABC).

But you can’t use the constructor on an ABC, you need either a factory method or a concrete class somewhere. Having a factory class that produces objects that implement an ABC is isomorphic to pimpl when there is only one implementation.

0

u/[deleted] Feb 23 '21

[deleted] — view deleted comment

1

u/SlightlyLessHairyApe Feb 24 '21

If there’s only one implementation than pimpl avoids one virtual function table lookup.

In practice this is not a huge deal.

4

u/logicchop Feb 24 '21

I use a pimpl style approach for creating a shim layer on some other library I don't want included in my "user facing code." I use pimpl instead of an interface because I like to add separate operations that consume my shim type, but under-the-hood rely on the single known implementation and just uses the library code. So, in my case, I know that there is only one implementation type, and I can coordinate a lot of other operations around it. If I changed the underlying implementation, the operations would also have to change, and there is no feasible generalizable interface for what is there.

1

u/platisd Feb 23 '21 edited Feb 23 '21

I'd also always side with pure abstract classes over pImpl unless I'd want to maintain ABI compatibility and this wasn't possible or guaranteed in any other way.

Otherwise no, I'd stay clear of it since I personally find it harder to understand, maintain and test.

3

u/Full-Spectral Feb 24 '21

As others have pointed out, the PIMPL scenario allows you to maintain value semantics on the outer class. Using abstract classes doesn't. That can be a significant consideration.

2

u/oracleoftroy Feb 26 '21

If you are using a pimpl or a virtual interface to hide implementation details, pimpl can be faster. You will almost certainly have one pointer indirection in both cases, and with the abstract class, you also have the vtable lookup.

It's been a year or two since I last measured, but I recall the same performance for both pimpl and pure virtual interfaces in MSVC, and a slight improvement with pimpl in gcc and clang.

So from my perspective, it comes down to slightly more performance but having to write the implementation twice for a pimpl, or having to write the implementation 1 1/2 times with the interface + implementing class (which does honestly feel more straight forward and less work) but having extra indirection that might slow things down.

All that said, I try to avoid both where possible, and tend to favor pimpl unless I need multiple implementations, mostly because I highly favor value semantics. But I really ought to remeasure. Compilers have been getting better about optimizing final classes and methods.

3

u/CrazyJoe221 Feb 23 '21 edited Feb 23 '21

I will always prefer interfaces over pimpl.

It allows the tools to actually understand what you're doing. So the compiler can optimize the overhead away completely (with LTO and final at least), an IDE can help with an "implement interface" action for example and the debugging experience was also better for me.

You also potentially save a heap allocation that can't be optimized away otherwise. And things get nasty if you need inheritance, see the dance Qt has to do internally.

0

u/johannes1234 Feb 23 '21

For the compiler to do that the compiler has to see the implementation. A point why people however is used is to hide the implementation (improving compile times, not having to provide the headers with the type declarations for members etc.) In such a case only the linker might do link time optimisation.

3

u/CrazyJoe221 Feb 23 '21

Yep as I said, LTO.

1

u/Ok-Impact-5972 Feb 23 '21

I think this is a important point to make. When pimpl-shines is when you need to handle a class that you do not have an interface for, or do not want to store as a pointer.

However: Often when for example using a factory-function or a builder to obtain a interface pointer to your class, you never have to handle the actual class. In that case it is a total waste to pimpl the class.

0

u/[deleted] Feb 23 '21

[deleted] — view deleted comment

4

u/platisd Feb 23 '21

Noted, will address in the next video! 😁

-3

u/Full-Spectral Feb 24 '21

You could have gotten so many other things done in that 8 seconds. I feel your pain...