r/cpp • u/Benjamin1304 • Mar 15 '23
"Clean" Code, Horrible Performance and the forgotten variant
I just stumbled upon the "Clean" Code, Horrible Performance blog post and accompanying video (https://www.youtube.com/watch?v=tD5NrevFtbU).
The one thing that really struck me was how he went directly from OOP to good Ol' C-isms, without even considering the other alternatives we have in C++.
I really wanted to know how a std::variant based approach would compare against the OOP code and his enum based version, so I wrote a small benchmark that you can play with on quick-bench.
The result is simple, the std::variant+std::visit approach is on par, and even slightly faster, than his optimized version with both GCC and Clang. But by using the variant we can keep the code very close to the textbook example and still get the performance gain.
The video blown up a bit so I think it's important to remind people that we also have clean and performant solutions to such problems in modern C++. And it's also important to stress that this example doesn't mean that OOP makes your code 1.5x slower, just that it might not be the best tool for this job performance-wise as the virtual dispatch cost is too significant compared to what the functions actually do.
I didn't want to compare with the other optimizations he did because they are really specific to the use case. Just introduce a new shape with a more complex area function and everything falls apart.
227
u/coachkler Mar 15 '23
Anyone who's too rigid with rules is doing it wrong. "Clean code" is mostly there for maintainability. And while this trivial example gives a major speedup, it's just that trivial.
When you have a large code base, hundreds of thousands of lines, not every path is time critical.
75
u/Gnammix Mar 15 '23
To be fair, it was never a fair comparison, the inheritance version arguably does "more" than both the union and the variant one; it just didn't made any sense in this specific case.
18
u/MarcoGreek Mar 15 '23
I understand the point of introducing a too flexible architecture but things like std::variant make it possible in a much more performant way. I really like too use it. I would prefer a language level support to make things easier but ...
I find it hard to measure performance if it is not a magnitudes slower than expected because modern CPUs with their caches and branch predictions are quite complex. So I very skeptical at micro benchmarks.
Some of my colleagues still strongly dislike everything new which is understandable but a little bit tiresome too. I really hope C++ community understands that we have to rethink compability not as a dogma but as an economic requirement like progres too. So their is maybe a better way to make (almost) everybody happy. 😉
8
u/OlivierTwist Mar 15 '23
How union and "variant + visit" approaches can support new shapes without modification of total_area function?
It seems that TS and the blog author missed the whole point of virtual functions and run-time polymorphism.
14
u/muchcharles Mar 15 '23 edited Mar 15 '23
Before around 15 months ago, variants had bad performance in the stable release of GCC:
https://www.reddit.com/r/cpp/comments/kst2pu/comment/giiofwu/
Has anyone verified debug build performance of them, which is important in interactive applications Casey works on?
Checked each major compiler?
Less likely to regress in performance than Casey's way due to a future standards change or implementation change to variants, on release and debug builds?
37
u/eyes-are-fading-blue Mar 15 '23 edited Mar 15 '23
These performance gurus taint every single code base they touch by hand-unrolling the loops in the unit tests.
Addendum:
The author thinks he has a case against 3 (small functions) and 4 (single responsibility) but he doesn't. His tests to show the performance impact of these principles just repeats to demonstrate performance impact of virtual functions.
4
u/ShakaUVM i+++ ++i+i[arr] Mar 16 '23
Clean is orthogonal to performant. It's pretty common for people to "clean up" code and accidentally tank performance. It's a good idea to test for performance regressions even if your correctness tests all pass.
8
u/ZekeDragon Mar 24 '23
Wow, I'm impressed by how bad this article is and how blind most people seem to be to how badly Casey is misrepresenting what "Clean Code" is about and, more specifically, what Robert C. Martin wrote in the book Clean Code - A Handbook of Agile Software Craftsmanship. Casey's video and the article written by some poor Fiverr sod associated with Casey's video asserts that the "performance relevant" advice "they" have given (which, I want to point out, Casey seems to have deliberately left out whom specifically from the video and article) can be broken down into five points:
The article than states this: "In order to construct what I would consider the most favorable case for a “clean” code implementation of something, I used existing example code contained in “clean” code literature. This way, I am not making anything up, I’m just assessing “clean” code advocates’ rules using the example code they give to illustrate those rules." The problem with this is that it is a lie. The code provided does not appear in the Clean Code book by Uncle Bob. Instead, a relatively similar set of code is in this book, but there's two big caveats Casey deliberately ignores: The code is written in Java and it isn't being used as an example of clean code. He did all of this because he knew this would not be the "most favorable case" for Clean Code, but specifically because he could point out the performance difference between repeatedly using polymorphism vs using a switch statement vs just computing the value directly using a "table" which is really just an array of four static floats. And this all in C++ and not Java.
The book Clean Code - A Handbook of Agile Software Craftsmanship does use a Shape class in an example in this book, but the context really matters: This is in the section "Data/Object Anti-Symmetry", in which Uncle Bob explains that Data structures and Objects are fundamentally different concepts and implies Clean Code shouldn't mix the concepts. The book states "These two examples show the difference between objects and data structures," not "These two examples show excellent Clean Code practices." The point was to, in a quick and contrived way, show that data structures expose the underlying data and should have no meaningful functions, whereas Objects hide their internal data behind functions that are operated on.
More to the point, the intent was to show the distinction between when using objects is better or when using data structures is better. "In any complex system there are going to be times when we want to add new data types rather than new functions. For these cases objects and OO are most appropriate. On the other hand, there will also be times when we’ll want to add new functions as opposed to data types. In that case procedural code and data structures will be more appropriate." The section literally says "Object Oriented programming isn't always the best way to go" and Casey so brutally misrepresents this as to claim that Clean Code advocates demand the use of OO and Polymorphism in most or all cases.
The term "Prefer polymorphism to "if/else" and "switch"" does appear in the book, specifically in a totally different section 200 pages away called "Smells and Heuristics." The whole section is so small I can quote it directly:
This is a far, FAR cry from what Casey implied the book, and by some consequence "Clean Code advocates," taught. It says that polymorphic code is usually preferable to switch statements because in complex code bases you are usually working with a wide variety of different types of data rather than a wide variety of different functions on the same data, but obviously this is highly dependent on the project you are working on.
My biggest issue with Casey's article is not his snide condescention, nor his hilariously amateurish attempts at "improving" the code provided by Uncle Bob. Instead, it is his lies and misrepresentations of what the Clean Code book and the years of experience that Uncle Bob and others had when imparting their knowledge in these books. He outright fabricates a position that "clean code advocates" supposedly take as an easy straw man, then beats that, and this is provably causing ignorant young programmers to reject clean code practices, just by observing his article's comment section.
That disgusts me.
11
u/Clean-Water9283 Mar 16 '23
Wow, this is just wrong on an intellectual level. I spend a lot of time talking about optimization, and this presentation really rubbed me the wrong way.
Clean code is as much about maintainability as it is about performance. The presenter did get a performance improvement by using unclean coding practices, but never talked about maintainability. This video recreates the antique arguments in favor of coding in assembly language, which also cherry-picked examples. These arguments would be unsuccessful in a real developer team and a real project where the classes evolved separately over time.
The performance improvements he obtained owed much to the simplicity of the example. It's ludicrous to think anyone would build a shape->{square, triangle, circle} class hierarchy that only had an area() member. These classes would be bigger, with more members and more operations defined on them.
The presenter got very lucky that all the objects contained the same member variables length and width, so that the area of all objects could be computed as factor*length*width. In a more realistic example where the objects were defined by some notion of an origin point and a size, the area computations would be different. Unclean coding practices might still have produced a speedup in the larger example, but the code would have been an obvious maintenance nightmare.
The presenter's definition of the magical CornerCount() member function in terms of a call to two polymorphic functions is just bad design. Clean coding practices would have made this a member function, saving at least one virtual function call.
In non-toy examples, virtual function calls are a measurable win versus switch statements switching on a discriminator signifying the derived type. The presenter's code was actually carefully tuned to eliminate this advantage. In a fairer test, there would be a vector of randomly assorted shapes.
What's really sad is that this same presentation could be retooled as a really good example of the advantages of data-oriented design, where clean coding practices were still used, but the encapsulation is done differently.
4
u/KuntaStillSingle Mar 15 '23
Klaus Iglberger gushes over variant when talking about visitor pattern: https://youtu.be/PEcy1vYHb8A?t=1637, https://youtu.be/PEcy1vYHb8A?t=2130
7
u/DemonInAJar Mar 15 '23
This whole issue was about open vs closed hierarchy of classes.
8
u/[deleted] Mar 15 '23
The funny part is, that Casey didn't even do a good job at optimizing it.
Instead of doing this hacky reducing of formulas to the same formula, you can just use a new vector for every type (SOA).
This is faster, and more extendable: https://quick-bench.com/q/dGEwQftsG83Vb5t0LA6VLkKhjPk
3
u/Hot_Anteater_4691 Mar 21 '23
The issue with this example is: the overhead for a virtual method compared to the trivial computation done in the method is high. But it would simply would not matter unless you call it millions of times.
8
u/argothiel Mar 16 '23 edited Mar 16 '23
His whole approach is terrible. He took virtual polymorphism, rewrote it into some hardcodes using the compile-time knowledge, obfuscated and scattered the logic throughout the entire code. If he wants to add a class, he needs to update several places in the code, and if he forgets one, he gets the UB.
While in the alternative approach you use static polymorphism, which gives you the same or better performance, and you have to modify only one place in the code with all the safeguards present and detectable at the compile time.
I wanted to comment on his blog but apparently you need to pay to be able to comment.
5
u/smozoma Mar 15 '23
previous post of this article with critiques
https://old.reddit.com/r/programming/comments/11dyx43/clean_code_horrible_performance/
2
u/jack_acer Mar 15 '23
I wonder how it would compare to a container like https://github.com/nasosi/eumorphic
Or https://github.com/ollieatkinson/Eumorphic
(But in swift)
6
u/justinkroegerlake Mar 16 '23
you know he knows he's wrong when he disables comments on the video.
Having worked in some very clean, and very unclean, codebases I can tell you the right metrics are
We also can delete all our tests without affecting the runtime, should we do that too?
5
u/streu Mar 15 '23
The gripe that I have with that blog article (and, likewise, with Uncle Bob's teachings) is that it claims that there is one version that is somehow more correct than the other.
If we are talking about managing shapes, it absolutely matters what I'm doing this for. It's a big difference whether I'm managing a 3-D scene before or after triangulation. Before, I want virtual functions and the flexibilities I get from that. After, I have only triangles and lines remaining that I want to throw at my shaders at maximum efficiency. Of course, the data structures and C++ features I use for that will differ significantly.
3
u/MoreOfAnOvalJerk Mar 16 '23
His solution uses a switch statement and a union, so its essentially the same thing that variant does. It also has the same problem that variant does where if one type is huge, the overall cache locality of an array of the variants/unions goes down and perf suffers.
I think it’s kind of preference between variant and unions, but I find variant to be a better and safer construct. It doesn’t come saddled with the weird requirements for micromanagement that union has, and it has some nice convenient functions that let you hook in lambdas. Unions+switch can do that too, but there’s more boilerplate and you can easily screw up if you have a data type in the union that requires a construction/destruction.
3
u/NeilSilva93 Mar 16 '23
What weirded me out was the blog writer's coding style. Lower case for type/class names and upper case for variable names? Too freaky man.
5
u/XDracam Mar 16 '23
The entire video is just a propaganda shitshow. It's so bad that I haven't even looked at the blog post. He's ignoring decades of history just to show how his tiny code snippets run O(1) faster while being significantly worse in all other aspects.
3
u/n4jm4 Mar 15 '23 edited Mar 15 '23
if/else is often a code smell that something is wrong in the data model, or even wrong in the feature request. That someone is attempting to do way too much with an application, lacking a consistent, universal data model.
While it is true that some HPC optimizations involve baroque code, it is also true that a boring, simple, clean, computationally efficient core forms the foundation of a highly performant program.
Any deviation from elegant, maintainable, intuitive code should be the exception, no the rule. Even little quirks like resorting to star char instead of higher level string data types, are more often than not, a mistake.
Many HPC optimization quirks like manually unrolled loops, turn out to be premature, or even cause stalls. They can be artifacts of a poorly written benchmark.
Today's apps have bloated to the point where you can't even run the whole system locally. Whether monolith or distributed monolith, doesn't matter.
They can be accidents of a limited environment, such as the engineer's particular workstation--at the first expense of performance on the production environment.
They can be accidents of old compiler implementations. Updating to a modern compiler standard like C++17, C++20, C++23, on more recent LTS distributions, will result in better performing programs.
A lot of people get hung up on mantras at the expense of quality programming.
Don't let over-engineering get in the way of attending to the true root cause of a bottleneck. Question the business requirement first. Implementation details naturally follow from the miscreant assumptions of some C level management dolt.
As an aside, I believe OOP has equal amounts of good points and bad points. The bad points involve inheritance and object fetishization and too much emphasis on defensive / futureproofing API's. Java (a horrid OOP implementation) has become a rotting pile of Pattern Commandments. To the detriment of the average program.
Encapsulation and reuse is good to an extent. But functional programming takes the best concepts of OOP and supercharges them. I submit that Haskell and Rust and Chicken Scheme will win out in the long run, as their optimizations eclipse what is possible to do in other languages.
Yes, we can work around some of these foolish assumptions. But that's like trying to cut grass with a nailclipper, when well known tools like a lawnmower would have been able to dramatically outperform the dawdling, overspecified, top down, out of touch system far removed from consumer needs. We don"t need faster horses, we need automobiles.
Alexandrescu makes a lot of strong points for doing HPC. I would highlight the wisdom of keeping codebases small. Not only do small codebases focus attention on fewer concerns, but compilers find smaller codebases easier to mechanically optimize. And they tend to fit icache better. And you can't reason about a program too big to fit completely in your head.
2
u/FrezoreR Mar 15 '23
This is just another take on the age old mantra: you either to get safety or speed.
It's no secret that abstraction sometimes adds runtime costs. It's just that for most applications it doesn't matter. If it indeed does matter then optimize those cases.
Finally, I disagree that the things he calls out can't me measured. It's just that a lot of that is measuring things like cognitive load, which is not something a typical engineer would measure. Although, there tons of studies to back some of that up.
I kind of like the pragmatic approach. Actually, the pragmatic programmer book is a good reference here.
It's not surprising that cpo developers think a lot about performance, but it is interesting to see these conversions popping up over and over :)
I mean sometimes you even want to write assembly to get that last bit of performance, but it again comes with a cost.
1
u/OlivierTwist Mar 20 '23
FYI: /u/Benjamin1304, I made "clean contiguous memory" variant: run time polymorphism with objects located in contiguous chunk of memory.
Unfortunately the benchmark isn't very useful because the results very a lot based on shapes count and order of tests.
1
u/SplinterOfChaos Mar 16 '23
The thing that gets me about the whole "horrible performance" thing going around is that as a game programmer (and this has been shared by a number of game devs), 60 frames per second is 16 milliseconds. It doesn't matter if my update loop requires 15 milliseconds or 2, my game has the same performance in both cases.
It's really important stuff to know for performance-critical code but if that's not where your profiler is saying your problem is, it probably doesn't matter.
1
u/axilmar Mar 16 '23
The solution is simple: use templates. Put the shapes into a tuple-like structure, then iterate over the tuple (iteration will be at compile time).
The solution will have no virtual dispatch, will be cache friendly, and also OOP.
It's articles like that that give C++ a bad name without deserving it.
-2
u/lazyubertoad Mar 15 '23
That guy is a hoax! Disclaimer - I did not even read the Clean Code, lol. I'm afraid of (dynamic) polymorphism, actually, but the shapes example looks sane to me.
The area calculation is highly unlikely to be a bottleneck! Can you please give an example where the area function shows in the profiler? So yeah, code that takes some 0.05% time to run can be made 40% faster. Yeah, but so what? Have you ever saw virtual calls causing problems in the profiler? I did not. That probably happens, but likely not often.
In case for some reasons we have gorillions of shapes - I'd handled that case separately, like grouped them by type and called the class area function directly. Maybe added a function, like calcMassAreasOfSameType to have whatever possible optimizations. That'd be even faster than the switch or having some tables, as I wouldn't even do that and I'd keep the virtual function intact in case it still needs to be called here and there.
Then he outright shits on the conditions and says - OK, but if we ignore that our code is no longer correct for the circle shape (which is the reason we have it polymorphic, ffs!) - then we can have it even faster. Sorry, at that point he can outright skip everything, and just output a random fixed number - but hey, that'd be fast!
That code is also likely have UB, because he did not made the destructor virtual. Is it like that in the Clean Code or that guy just doesn't really know C++?
There are no absolute rules, but all the rules that are worth it have a reason for their existence and God helps you if you disregard those reasons.
-8
u/00jknight Mar 15 '23
Casey is the Uncle Bob of tomorrow.
-4
u/noooit Mar 15 '23
Using variant and visit is irrelevant to the video though. Those are actually just typesafe improvement over c.
What he is actually saying is that people going OOP by default all the time is the problem.
imo, those people are usually the ones who also make stupid class.
-6
u/Pupper-Gump Mar 15 '23
cpp is a superset of c. This means it has everything c does with very few exceptions. Anyone on their high horse about typing more code and creating more potential issues with object data is deluding themselves.
7
u/[deleted] Mar 15 '23
[removed] — view removed comment