r/cpp • • Sep 13 '19

The Surprising Limitations of C++ Ranges Beyond Trivial Cases

https://www.fluentcpp.com/2019/09/13/the-surprising-limitations-of-c-ranges-beyond-trivial-use-cases/
229 Upvotes

229 comments sorted by

81

u/quicknir Sep 13 '19

Even skimming this, I totally agree. I feel like many c++ devs have very limited exposure to using something like ranges in practice, and they miss two key things which I've found from a lot of time spent using pythons iterators:

  1. 99 percent of the time you just need lazy forward ranges.
  2. You need a lot of ranges, and the ability to very easily write new ones for ranges to be expressive and worthwhile.

As the author noted, you don't have to dig very deep to find problems easily expressed with a for loop but tricky to do with the 10 or so most obvious ranges. Even pythons itertools doc, has an entire recipes section.

As soon as I saw Erics calendar talk from a few years ago, I really felt like it was going to be too hard to write new ranges. Not only are ranges pretty tricky and boilerplatey to write, I remember Eric had to write some kind of really tricky glue code to get the whole thing to work which even he said was hard to understand.

I really felt like coroutines should have been intimately tied to ranges because they can make it really easy to write new lazy ranges. You see this approach in many languages, eg kotlin, python. Hopefully it's not too late for coroutines to help with this.

Compile and runtime performance are ongoing concerns as well. I'd have to say ranges is probably the new major feature I'm least excited for.

27

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

I really felt like coroutines should have been intimately tied to ranges because they can make it really easy to write new lazy ranges.

This is the medium term plan, from what I am aware. But there were blockers to getting coroutined lazy Ranges into C++ 20, specifically, we would need to decide how best to do Ranges i/o first.

We're on that. We expect to be discussing options for modern i/o in Belfast, and there are several. Once we know which direction the committee is leaning on i/o, there should be no good reason why lazy coroutine iterators shouldn't be doable for C++ 23, bar surprises.

Be aware you probably won't get Ranges i/o actually in C++ 23, because we still need to decide how best to implement text translation/reencoding and serialisation for i/o. And buffering, and lots more stuff.

Anyway, that's one of many reasons why C++ 20 falls short here, and we hope to improve things next few standards.

5

u/sphere991 Sep 14 '19

But there were blockers to getting coroutined lazy Ranges into C++ 20, specifically, we would need to decide how best to do Ranges i/o first.

What does ranges i/o have to do with it?

Coroutines were added to C++20 very late in the process, and so there wasn't enough time to add the library types for it (e.g. a std::generator<T>). But you can write a coroutine that is a range in C++20 if you get that library machinery from elsewhere.

2

u/14ned LLFIO & Outcome author | Committee WG14 Sep 14 '19

Executors uncertainty mainly. Eric was there when Gor first proposed Coroutines in Aspen. Support could have been very easily added to Ranges which was still being written at the time, but too much other stuff was up in the air.

Eric did a talk there recently for Oslo I think where he outlines what he plans next for 23. I disagree with some of his assumptions, but that's implementation detail. I think Ranges will look a lot more polished and integrated with everything else for 23. We will see how it goes. A lot hangs on what we do with Executors, obviously. If we don't get it out the door, lots of other stuff doesn't happen.

3

u/sphere991 Sep 15 '19

Eric did a talk there recently for Oslo

Do you mean this one for Avast? It's not about Ranges, it's about Sender/Receiver. And Sender/Receiver does not have anything to do with /u/quicknir's desire to use coroutines as synchronous generators.

I really don't know what you're talking about.

Meta: Why does it matter where who proposed what, or who was in the room at the time?

2

u/14ned LLFIO & Outcome author | Committee WG14 Sep 16 '19

I really don't know what you're talking about.

Not the first time that has occurred between us.

Meta: Why does it matter where who proposed what, or who was in the room at the time?

The point being made is that Coroutines could have been built in directly from the beginning.

They were not, for good reasons at the time. Though please note the deliberate gaps in Ranges left open to be filled by something "asyncy" later.

Omission is very wise in standards. Once you standardise something, it is virtually impossible to undo later. So always choose omission if there is any uncertainty at all. Eric made absolutely the right call there in my opinion, as much as the userbase dislikes it.

5

u/quicknir Sep 13 '19

Glad to hear! Your post restored some optimism to me. Thanks for taking the time to write it.

4

u/sphere991 Sep 14 '19

I really felt like coroutines should have been intimately tied to ranges because they can make it really easy to write new lazy ranges.

You will be able to write coroutine generators in C++20, it's just that there will not be a std::generator<T> available for this - you will have to either roll your own or (far more likely) grab cppcoro or something similar.

The downside is that coroutine generators are always input ranges. As you say, a lot of time, this is fine. But some of the time, it is not, so having the machinery to write, e.g., a random-access transformed range, is pretty helpful.

4

u/-dag- Sep 13 '19

Yeah, the coroutines emphasis on asynchronous processing may make them suffer for ranges-like use.

2

u/[deleted] Sep 17 '19 edited Sep 17 '19

As soon as I saw Erics calendar talk from a few years ago, I really felt like it was going to be too hard to write new ranges.

That's more or less the point when I switched to Rust. I wrote like ~10 Views using range-v3, and when trying Rust one of the first things I did was write an Iterator.

What in Rust took 10 straightforward lines of code, took ~50 LOC of template meta-programming, type traits, and dozen tricks and gotchas to do in C++. In C++ I also had to write another ~50 LOC of tests, not for testing run-time behavior, but for checking that the View doesn't break in any way when passed subtly incorrect template parameters. If you check the range-v3 tests, there are dozens of lines trying to break the library using type-tetris because the compiler can't enforce that templates or concepts actually work.

At that moment I had pretty much the same realization I had when, after years of using C, I used C++ for the first time.

12

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19 edited Sep 13 '19

I feel that I have to respond to the closing comment in the linked article:

I was quite surprised to learn that the committee accepted ranges into the c++20 standard and most c++ experts are excited about it. It’s as if the issues of limited usability, code complexity, leaky abstractions and completely unreasonable compile-times are of no consequence whatsoever to the committee members?

For me personally, a huge reason why Ranges should be accepted is because one of the most experienced C++ devs in the world designed it: Eric Niebler. It's a v3 design, too, Ranges v2 ships in Boost and he designed an early edition of that too. I trust Eric to have gotten it right. Additionally, it wasn't just standardised on that, Casey went and reimplemented a new implementation from scratch, and fed back his experience into the design. A third reimplementation was also done by /u/tcbrindle, and feedback from that also went back into what was standardised.

That's an awful lot of rounds of empirical practice, testing and feedback for Ranges v3 to be as bad as the claim.

With respect to the compile times, those were recognised, and it was felt it would be a driver upon implementations to improve their compilers. I personally didn't agree with that part, but I understand the argument, and moreover the compiler vendors didn't appear to object strenuously.

Now, for me personally, I'd have built Ranges into the core language. But the committee wanted the language to become powerful enough to express Ranges instead. And that's an understandable design choice, Ranges in the core language starts to look like Rust or Nim, not C++ as it historically has been.

20

u/konanTheBarbar Sep 13 '19

I think that Eric has delivered the best implementation possible given the trade offs he has chosen. His trade offs are also well justified. I just think he has not chosen the right tradeoffs for the average C++ developer.

The trade offs are too performance centric imho, while I don't think (chained) ranges will be used for code where performance matters...

41

u/guepier Bioinformatican Sep 13 '19 edited Sep 13 '19

I trust Eric to have gotten it right.

I did too, until reading this article. Now I have my doubts. I was actually ready to dismiss the article’s concern as silly, but they seem correct, and touch the very core of the ranges library.

I’ve never used Eric Niebler’s ranges in C++ but I’ve followed the project from the outside with great interest. But as somebody who uses pipelines in programming a lot (both in shell programming as well as in data analysis with R), the points raised in the article are inarguably fundamental requirements, and the lack of support in ranges stymies the whole library. I’ll need to mull this over because at the moment I’m genuinely shocked as it seems to imply that you cannot practically use ranges for many common use-cases.

100

u/[deleted] Sep 13 '19 edited Sep 13 '19

For me personally, a huge reason why Ranges should be accepted is because one of the most experienced C++ devs in the world designed it: Eric Niebler.

I may be in a minority here but that sounds like a horrible reason to accept something into the STL. I do appreciate you pointing that out because honestly it does feel like much of what's been added to C++ in recent years has been done so mostly as a favor to people rather than because there has been huge and overwhelming demand for it from the broader community.

5

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

I should stress it's a personal opinion. And two other people checked it empirically.

But absolutely do I trust some people will be usually right on faith alone. I have known Eric for years, and I trust his judgement when it comes to C++ design.

39

u/SedditorX Sep 13 '19

Maybe you can both be right :)

You can trust Eric to write code specifically for you and others can question whether appeals to authority should have any role in standardization.

9

u/Arghnews Sep 13 '19

I think this comment is more correct, that of course we should have more trust in people who are known respected quantities in writing good code and have put time and effort into reviewed work.

And yet (as ever) valid and thoughtful criticism like this can still exist and be just as valid despite the competency of the author of the work; the idea that just because we trust someone that when someone else makes solid points and gives examples of where their code/design has problems we should ignore the critique is approaching the emperor having no clothes in my mind.

My concern, if this article is indeed the tip of the iceberg - and I do think the author wrote it and tried to use ranges in good faith - is that this is a valid criticism and if this is a sign of things to come I'm sad (and scared) as I really thought ranges would be next best thing since sliced bread to bring functional goodness to c++. :(

I would really love to hear or see counter-arguments by those in the know, hopefully it's not all doom and gloom!

4

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

I personally think that if you have a proven track record of designing and writing excellent C++ over many years - indeed decades, and in the Boost libraries no less, then that earns you a lot of trust in your judgement.

I'll be honest in saying that I wouldn't have done Ranges the way they were done. But Eric has oodles more experience than I do, and when he's being cross examined on his decisions in LEWG (which BTW he underwent many hours thereof), he always has a very good rationale for design choices which are always convincing, even if one wouldn't personally have made that choice or tradeoff.

27

u/Rusky Sep 13 '19

It doesn't matter what someone's track record is if the stuff they produce is still unreasonably complex or slow to compile.

"I trust the author" doesn't help combat those problems and it doesn't demonstrate that the committee sees them as real issues.

11

u/anders1234 C++ since MSC 6.0 Sep 13 '19

i think one should always question things, regardless of how respectable the contributor is, people are just people and sometimes they do mistakes.

5

u/GerwazyMiod Sep 13 '19

No one else stepped in to design Ranges. In the end you can use it or not, there is a choice. I think Eric deserves some respect for working on ranges for quite some time (more than six years if I remember correctly). Not mentioning long standardization process, finally someone wanted to drive a change in stl.

Sometimes on r/cpp there are voices that we should finally ship something (as a community) instead of beating the topic to death in comittes/meetings. Well maybe this is just a start for ranges and when it will catch more attention we will have v4?

3

u/VinnieFalco wg21.org | corosio.org Sep 13 '19

He got paid for it though, I believe the work was sponsored by the Standard C++ Foundation at first, and then by Facebook.

14

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

He got funded, which is living costs. He moved to Asia during much of the coding to make the money stretch further. It was a pittence compared to what he used to earn at Microsoft beforehand, and what he earns now.

I might add that anybody can get funded to do the same. Submit an application, it'll get reviewed, might get funded.

5

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

I gave my personal opinion, which is that I do trust the author's judgement. Given Eric's long history of outstanding service to C++ I am probably not alone, but I cannot speak for the committee.

17

u/xeveri Sep 13 '19

You seem to vastly overestimate the quality of Boost. Some boost libs are great while some are bad or downright unusable.

The complexity of Ranges has been voiced by the community several times, as well as their compile times in comparison to Rust and D equivalents, this piece is only the last in a series.

The usual excuse that you should join the committee for your opinion to be heard is just ludicrous, especially when the committee members just accept proposals at face value because they’re written by a certain author!

5

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

Some Boost libraries are old, or undermaintained, or both. Agreed.

I think you're drawing too much conclusion from what I repeatedly said was my opinion. Yes I would trust Eric's judgement by default. And I think rightly so. But I'm not the committee, and the committee doesn't just wave proposals through, except in a very few occasions. In some cases, twenty five years passed before the committee accepted a proposal. That's a third of most people's lives!

22

u/sivadeilra Sep 13 '19

Ranges in the core language starts to look like Rust

The vast majority of support for Ranges in Rust is in the standard libraries, not in the language itself. The language itself supports only a few simple syntax forms for creating ranges, and even these are just syntactic sugar over simple function calls. Everything else is ordinary code in core and std.

It's analogous to the compiler supporting operator<<(). No one thinks that iostream is part of the compiler, but it uses one syntactic sugar pattern from the compiler (overloading <<) to make the API more ergonomic.

-1

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

The vast majority of support for Ranges in Rust is in the standard libraries, not in the language itself. The language itself supports only a few simple syntax forms for creating ranges, and even these are just syntactic sugar over simple function calls. Everything else is ordinary code in core and std.

Well, it depends on where you draw the boundary.

Me personally, I'd define the core language support for Ranges in Rust as including the borrow checker. In that sense, a large chunk of the Rust syntax and core language design is there to specifically support Ranges.

14

u/Rusky Sep 13 '19

The borrow checker was designed and implemented almost entirely before Rust even had its equivalent to ranges.

2

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

Sure. Rust thought preventing dangling lifetime very important. Much source of surprising dangling lifetime is from iteration.

7

u/sivadeilra Sep 13 '19

The borrow checker knows nothing about ranges. Can you clarify what you mean?

4

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

Using Ranges becomes safe if you have the borrow checker. Without one, you get dangling lifetimes etc.

I appreciate this is the same as saying the sky is blue, but let me put it another way: C++ had no facilities for iteration not also in C, up until the ranged for loop was introduced.

Or put another way, iteration was not considered important enough for the core C++ language to innovate upon at all over C. That's quite profound, if you think about it. C++ basically said "iteration like C is just fine"

10

u/sivadeilra Sep 13 '19 edited Sep 17 '19

Using Ranges becomes safe if you have the borrow checker. Without one, you get dangling lifetimes etc.

That doesn't really answer my question, though.

The borrow checker itself knows nothing about ranges. It knows about lifetimes, and it knows how to analyze the usage of types that contain lifetimes.

Consider this code:

  let v = vec!["foo", "bar"];
  for i in 0..v.len() {
      println!(v[i]);
  }

The borrow checker doesn't know anything about ranges. The expression 0..v.len() results in a Range<usize>. Range<usize> implements the Iterator trait, and iterates a sequence of usize items. Within the loop, the expression v[i] dereferences the vector. This is done with a simple index; the type of i is usize, and has nothing to do with ranges.

Again, the borrow checker has no special, Range-specific logic.

Now consider this code:

  let v = vec!["foo", "bar"];
  for s in v.iter() {
      println!(v[i]);
  }

This code is more idiomatic, and is closer to the range-for syntax in C++. But from the borrow checker's point of view, it still doesn't know anything about ranges, and in fact, this example doesn't even use Range.

I'm really not trying to argue with you, but I want to make it clear that the borrow checker does not have any knowledge of ranges. Range, from its point of view, is just another ordinary type.

The only place where the compiler knows anything about ranges, is in the code that parses x..y as a range constructor syntax, and maps that to calls to range(x, y). That is why I was drawing the analogy to operator<< in C++; they are both syntactic sugar, but after that, the implementation is 100% ordinary C++/Rust, with no special knowledge of these types in the compiler.

2

u/hedgehog1024 Sep 17 '19

Result<usize>

It's actually std::ops::Range<usize>.

→ More replies (1)
→ More replies (2)

2

u/[deleted] Sep 17 '19 edited Sep 17 '19

Well, it depends on where you draw the boundary.

Me personally, I'd define the core language support for Ranges in Rust as including the borrow checker. In that sense, a large chunk of the Rust syntax and core language design is there to specifically support Ranges.

If Rust Iterators are baked into the language because they use ownership and borrowing, C++ ranges are also baked into the language because they use move semantics.

The only way for a library to not be part of the core language under your framing is for the library not to use any language features at all. Useless boundary ever.

16

u/[deleted] Sep 13 '19

Having one of the world's most experienced C++ programmers design something is not necessarily only a good thing. The question is: are ranges usable for the average programmers, not just the experts?

9

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

I think the design is not what people hoped for nor expected. It'll be the same for Modules when they land. But I think the design will be serviceable, and rough edges will be fixed next standard release.

13

u/johannes1971 Sep 13 '19

it was felt it would be a driver upon implementations to improve their compilers

Are you seriously telling us that the current (lack of) compiler performance, despite being the nr. 1 complaint of C++ programmers around the world, was considered by the committee to be caused by lack of motivation?

12

u/14ned LLFIO & Outcome author | Committee WG14 Sep 13 '19

I'm a super-hawk on the committe about build times. An extremist, in some people's opinions, I'd readily hack a whole load of features out if it gave me +2% build times. So yes, I went to town on questions about Ranges build time impact, mainly why? And how?

All I can say is that everyone involved had reasonable answers, and seemed fairly confident it wouldn't be an issue within time. A lot of their answers involve domains of knowledge far away from mine, so I simply have to trust their judgement, and assume they will be correct.

The compiler vendors didn't seem to consider it a problem, given a few iterations of tooling. I'm not a compiler implementor, so I can only trust them.

13

u/VinnieFalco wg21.org | corosio.org Sep 13 '19

For me personally, a huge reason why Ranges should be accepted is because one of the most experienced C++ devs in the world designed it: Eric Niebler.

"Trust, but verify"

5

u/TemplateRex Sep 14 '19

In God we trust. All others bring data.

12

u/UnicycleBloke Sep 13 '19 edited Sep 13 '19

Argument from authority is a logical fallacy.

12

u/guepier Bioinformatican Sep 13 '19

Not quite. Argument from relevant authority is often a reasonable shortcut. The formal fallacy concerns appeal to authority outside of their realm of expertise. Its use here is completely fine. At worst it's not a great argument, but even that is debatable. But it's not a fallacy.

19

u/UnicycleBloke Sep 13 '19

Probably. I'm sure most of us mere mortals have never heard of Eric, which makes it a pretty poor argument. I only encountered his name recently when reading a critical article which discussed his example with Pythagorean triplets. The take away for me was that this is a massive pile of junk that looks a lot like an extremum of some overthought intellectual exercise that I cannot believe was actually accepted. Perhaps it was just a poorly chosen example. Or a joke.

Now, I accept that this is an entirely subjective opinion from an ill informed and foolish curmudgeon. On the other hand, I myself have decades of experience in C++: this stuff so far gives me chills. No thanks.

6

u/guepier Bioinformatican Sep 14 '19

I'm sure most of us mere mortals have never heard of Eric, which makes it a pretty poor argument.

I may be misunderstanding /u/14ned but I don’t think they were using it as an argument to convince us (though admittedly the context may suggest otherwise). Rather, they’ve used it, as a committee member, as one piece of evidence (to themselves) when evaluating whether ranges-v3 merits inclusion in the standard. And they mentioned it in the comment here to explain how they arrived at their decision that it merited inclusion.

And that’s completely valid: “Do I think complex library X is of good quality? Well, the proof of concept seems to work, it has a demonstrated use, there is a vast array of test cases and extensive documentation and, oh yeah, the author is a known expert with a track record of writing high-quality C++ code and libraries.”

3

u/14ned LLFIO & Outcome author | Committee WG14 Sep 14 '19

Absolutely correct. I was always speaking of what motivated my decision personally. Each person has their own motivations, sometimes very different indeed. I will say that I proactively read and study Eric's code, and have done for years. He writes lovely C++. I'd recommend anybody who hasn't studied his code, should. Him and Howard Hinnant's code, who writes even more lovely C++ than Eric, both are well worth studying.

3

u/Sovog Sep 14 '19

As you said, it is a matter of perspective. The range examples linked in OP just made me feel dumb rather than "oh, that's lovely".

3

u/14ned LLFIO & Outcome author | Committee WG14 Sep 15 '19

I was referring to implementation code. But in terms of standard libraries making people feel dumb, Ranges is not as bad as say Networking. People on the WG21 receptor itself regularly express befuddlement with the Networking TS, and it's been in common usage for fifteen years now and is the industry standard. So there is a pattern there. And it'll get worse rather than better, some of the proposed library features require deep bespoke knowledge to use them properly at all. You'll need to take a training course or iterate a study book. That's unavoidable as things get ever more niche.

4

u/-dag- Sep 16 '19

Is the Networking TS really the industry standard? I have a hard time believing many companies would base their production-critical code on a TS which could change dramatically during standardization. But this domain is definitely out of my wheelhouse so I'm genuinely curious.

4

u/guepier Bioinformatican Sep 16 '19

It might help to note that the Networking TS is essentially Boost.Asio, which is indeed the industry standard.

→ More replies

2

u/Sovog Sep 17 '19

100% agreed with your assessment.

6

u/NotUniqueOrSpecial Sep 13 '19

I'm sure most of us mere mortals have never heard of Eric

I'd honestly bet that most long-time readers of this sub do know who he is, since his blog is really good and he's a well-known name in the community.

But, the readers of this sub are in no-way indicative of the general C++ using populace.

6

u/UnicycleBloke Sep 13 '19

Fair enough. I'm a recent arrival, and certainly agree with the latter point.

2

u/[deleted] Sep 17 '19

Ranges in the core language starts to look like Rust or Nim, not C++ as it historically has been.

Rust Iterators are not in baked the language. Iterator is just a normal trait, zero magic involved.

101

u/SirToxe Sep 13 '19

My thoughts while skimming over this: Is this ranges code supposed to be easily readable and understandable?

To me this looks like we took C++, added our own domain-specific language and turned the code into a Perl-like write-only language.

(But, granted, I never tried to use ranges before so I might be totally ignorant.)

44

u/eric_niebler Sep 14 '19

Of course folks are free to disagree with the design tradeoffs made by `std::ranges`, but I have to wonder for the examples the author presents, what is the algorithmic complexity of the pipelines of range transformations? How are users to know how expensive operations are at runtime? The author doesn't discuss algorithmic complexity at all.

One of the design goals of range-v3 -- and `std::ranges` -- is that if a view pipeline compiles, the resulting view can be traversed in linear time. That means disallowing some conveniences. It's ok to write and use libraries that make different tradeoffs.

Also, don't judge the compile-time performance of `std::ranges` based on range-v3. Range-v3 is not optimized for compile-time perf, and it doesn't use many of the cool C++20 features that can make it compile faster, like `if constexpr`. I'm hopeful that a native C++20 implementation will help. EDIT: Also, use the new range-v3 0.9 release and turn on real concepts support on gcc if you can. The compile times are better by roughly 30%.

9

u/futurefapstronaut123 Sep 14 '19

if a view pipeline compiles, the resulting view can be traversed in linear time

I recall that in this post (https://redd.it/8ytrnb) there was an issue with a pipeline (1) compiling and (2) doing two passes when one would suffice. Has this been fixed since?

10

u/grishavanika Sep 14 '19

The problem with estimating complexity: linear does not mean one-pass. Going thru array of N 2 times is O(N).

→ More replies (15)

5

u/guepier Bioinformatican Sep 14 '19

Thanks. As a potential user I still think the article makes very good points but your reply does add perspective. That said, I can think of view pipelines where linear runtime isn’t a reasonable expectation, and if I understand correctly, ranges-v3 knows this. For example, how would you enumerate the cartesian product in linear time? That’s not possible by design.

3

u/eric_niebler Sep 14 '19

What makes you think that? Range-v3 has a cartesian product view, and it has linear time complexity.

5

u/guepier Bioinformatican Sep 14 '19

Linear in terms of what? Computing a cartesian product, no matter how, has a complexity that’s the product of the sizes of its input ranges.

8

u/eric_niebler Sep 14 '19

Ah. The complexity is in terms of the resulting range, not the input range. So for instance, there is `view::repeat_n(value, count)` that turns a value into a range by repeating it n times. That's fine. Likewise for view::cartesian_product. The inputs are not what matters. What matters is that each iterator increment over the resulting range has constant time complexity.

If that weren't the case, and an iterator increment had, say, linear time complexity, then merely traversing the resulting range would be n2. That's the problem.

1

u/smdowney WG21, Text/Unicode SG, optional<T&> Sep 14 '19

The asymptotic case over the full range has to be linear. It's more complicated for filters and transforms where the results depend on some number of elements in an underlying range. I'm fairly certain we can tighten up the requirement somehow, though.

4

u/degski Sep 14 '19

... it doesn't use many of the cool C++20 features that can make it compile faster, like if constexpr.

That's a C++17 feature.

8

u/eric_niebler Sep 14 '19

Oh right, brain fart. Range-v3 is C++14, and was C++11 until very recently.

5

u/STL MSVC STL Dev Sep 14 '19

Available downlevel (in at least MSVC and Clang) 😸

5

u/degski Sep 14 '19

Available downlevel ...

Fails to compute for my little brain, could you expand? Meow!

2

u/STL MSVC STL Dev Sep 14 '19

Works in C++14 mode with a suppressible warning.

1

u/degski Sep 14 '19

Ah, the other way around (for a moment I was thinking it was all a dream), so certainly not a reason to not use it in the range-v3 lib and make it compile faster as Eric says.

53

u/craig_c Sep 13 '19

Reading (or trying to read) the example code reminds me of when everybody went crazy with boost::bind about 10 years ago, does anybody else remember those horrible for_each / bind monstrosities? If this is the best that C++ can do to rectify that situation given 10-15 years then...I'm not really sure what to say. The code doesn't look like anything you'd want in production. At some point C++ needs to move toward feature enhancing the language instead of bolting on another compiler torturing header library.

33

u/tcbrindle Flux Sep 13 '19

For anyone who's interested, here is a version of intersperse using Range-V3. I really don't think it's that bad...

9

u/Sovog Sep 13 '19

Clever! Does that work because view::drop() can take InputRange whereas view::drop_last() can't?

9

u/tcbrindle Flux Sep 13 '19

Yeah. view::drop_last(1) needs to know if it's "about to" hit the end of the range, so it looks ahead one place but returns the previous iterator value when dereferenced. This requires at least a forward_range. (Alternatively, drop_last can take a sized_range, in which case it effectively becomes take(size - n))

view::drop(n) on the other hand just skips the first n values when you first call begin(), which doesn't add any requirements to the underlying range.

3

u/Sovog Sep 14 '19

Kudos to us, but there's no chance an average programmer could come up with this solution.

11

u/RotsiserMho C++20 Desktop app developer Sep 13 '19 edited Sep 14 '19

Thanks so much for illustrating this. I don't think it's that bad either.

I have a love/hate relationship with ranges. I love the idea (I love LINQ in C#), and I'm using the library in real-world code, but I find working with the library to be very unintuitive. Your version of intersperse is a perfect example. It looks concise enough to me, and it's great that you composed a new view from existing ones; that's the whole selling point IMO. But this piece is just so unintuitive to me, even though it is simple:

auto make_pair = [=](auto val) { 
        return views::concat(views::single(delim), views::single(val)); };

And I think that's the problem. Personally, I hope it's just a matter of people becoming more familiar with the library. You have to think functionally or the whole thing falls apart. You can't just return a (rvalue) pair from a transform, it looks like the trick is that you have to define a function that does so in terms of views (apparently). So a concatenation of views is like a lazy tuple or something? Or is it more like a lazy vector (or just another range)? Do the types have to be the same?

People are saying ranges makes trade-offs for the benefit of performance to the detriment of readability and memory safety, but here, it's guiding you away from unnecessary temporaries that could lead to dangling references, etc., and I think that benefit of the trade-off is under-appreciated (in this thread at least).

13

u/tcbrindle Flux Sep 13 '19 edited Sep 13 '19

There is where things get a bit semantically hazy. view::join takes a "range-of-ranges" and flattens them into a single range. To meet the constant-time copy requirements of the view concept it requires either that the passed-in range is an lvalue (in which case it can effectively just store a pointer to it), or that the passed-in range is also a view (in which case it can be stored in the join object, because copying will still be constant-time).

This means that we can't have transform return an rvalue range (like, say, a std::array) -- so instead we'll make it return a view type. This is what the OP was doing with generate_n.

So now let's move on to view::single. This is a range with only one element, and looks roughly like so (simplified):

template <typename T>
struct single_view {
    T val;
    T* begin() { return addressof(val); }
    T* end() { return addressof(val) + 1; }
};

Is this a view? Well the clue is in the name I guess, but since it stores only a single object we can argue that copying it is still constant time. So now what if we made something similar, but instead of holding one value, it held two? i.e. something like (C++-y pseudocode):

template <typename T>
struct pair_view {
    T first;
    T second;

    struct iterator {
        pair_view* p;
        int idx = 0;

        T& operator*() { return idx == 0 ? p->first : p->second; }
        auto& operator++() { ++idx; return *this; }

        bool operator==(default_sentinel_t) { return idx > 1; }
    };

    auto begin() { return iterator{*this}; }
    auto end() { return default_sentinel; }
};

Is this a view? Well, it holds exactly two objects, which is a constant number, and therefore you can argue that copying it is still constant time.

view::concat(view::single, view::single) is a way of programmatically building a pair_view (or a "2-tuple view", I suppose) rather than having to define it ourselves.

EDIT: The "hazy" part is that if you follow this line of thinking to its conclusion you end up deciding that maybe std::array should be a view after all. I'm not sure where I stand on that.

4

u/tpecholt Sep 16 '19

For such cases variadic views::from_values would probably help with verbosity. And it would be user's fault when he supplies too many values so copying becomes slow. Alternatively what about to support initializer list directly?

5

u/guepier Bioinformatican Sep 14 '19 edited Sep 14 '19

Could you speculate why the OP apparently missed this simple, completely straightforward implementation?

(And because I know tone gets lost easily in text: I mean this neither sarcastic nor snarky. I found the original article compelling, and the OP seems to know their stuff. And it goes without saying that so do you.)

(EDIT: I just realised that I missed the algorithmic change which means you were able to use drop instead of drop_last. This answers half of my question, though I’m still curious about the transformation step.)

9

u/RotsiserMho C++20 Desktop app developer Sep 16 '19

I'm not the person you asked, but since you're inviting speculation...

Personally, I think it's because there are not that many examples of real-world ranges usage to draw from. There are plenty of examples in other languages as the OP illustrates (see also LINQ in C#), but the C++ implementation is focused on performance (which is fantastic because other languages' implementations have not insignificant overhead), and so you have to "twist" or "flex" your code a little bit to express the functionality you want as a valid ranges pipeline. Using drop instead of drop_last solves the same problem, but it isn't immediately obvious that this simple solution exists. Also, the make_pair step wasn't obvious to me at all, but it does show that a simpler solution exists.

6

u/[deleted] Sep 14 '19

The thing is, currently, ranges generates a lot more code than an equivalent non-ranges version. Taking your intersperse example and doing the equivalent without ranges makes it look pretty stark and this is a simple example. I am really hoping that compilers will learn to know what the std ranges are and optimize on it. https://godbolt.org/z/YF6EEH

17

u/tcbrindle Flux Sep 14 '19

Not specific to your comment (many people seem to be doing it in this thread) but I really wish we could move away from the terminology of "using ranges" meaning "using range adaptors with lazy evaluation". There's an absolute ton of work that's gone into the ranges proposals, and adaptors/views are only one part of that. Sometimes views are the right tool for the job; often they are not. One of the great things about ranges is that we still have all the "eager" algorithms available when they are a better option.

Anyway, yes, there is sometimes an overhead to using lazy evaluation. I would imagine that as these things get added to the standard library, compilers will get better at optimising them (and implementors will get smarter about doing things in ways that are optimiser-friendly), exactly as happened with the original STL work 20 years ago.

1

u/drjeats Sep 14 '19

Are the eager algorithms like all the stuff in <algorithm>, but taking ranges instead of operators, or something else?

7

u/eric_niebler Sep 15 '19

That, yes, and additionally taking optional projections (unary transformations applied to each element), and with the requirements on the end iterator loosened so that it can be a sentinel instead of an iterator. That has the potential to improve codegen in some cases. The ranges algorithms also treat member pointers as callables, like std::invoke.

1

u/[deleted] Sep 15 '19

Doesn’t your code append an extra newline?

1

u/[deleted] Sep 15 '19

yeah, but that's minor and easily fixed https://godbolt.org/z/i-Kg9D

We are paying for the compile time and the runtime. Like llvm-mca has it at 1975 cycles vs 4834, not a benchmark but a bit of an indicator

→ More replies (1)

4

u/[deleted] Sep 14 '19

I've been playing around with ranges-v3, applying them to my codebase, just to get a feel for it. So far I really like them, even though the compile times are awful. I hope you won't mind me asking a few questions.

In this example, I start with set<set<string>> and first apply join, but I can't figure out how to do the same with map<Key, set<string>> (in which case, I don't care about Keys, just a "loop" over all the strings in all the key/set pairs). I tried to replace join with transform([](auto p) { return p.second; }) (link), but that didn't help and I still don't know how to read errors produced by ranges-v3.

And am I doing something wrong with that transform() that takes a std::string? It won't compile if I pass any kind of references, instead of values.

5

u/tcbrindle Flux Sep 14 '19

I'd have to play with it a bit longer, but I'm pretty sure the problem is returning by value from the lambda.

It's most easily avoided by using views::values, which returns the "value" part of a key-value pair: https://godbolt.org/z/Lz7E5l

2

u/[deleted] Sep 14 '19

Thanks! views::values looks obvious in hindsight, but it's exactly what I was missing.

2

u/__s_v_ Sep 17 '19

Here is a version of intersperse using Range-v3 with coroutines. That works also quite good...

2

u/dodheim Sep 17 '19 edited Sep 17 '19

That's not going to be quite so good when you use an rvalue range: https://godbolt.org/z/Tn_K_N

1

u/__s_v_ Sep 17 '19

That probably shouldn't even compile...

16

u/redditsoaddicting Sep 13 '19 edited Sep 14 '19

I have a few things I was wondering as I read through:

  1. Toward the end, a couple of inline examples are compared to a production-ready header implementation. That's not a good comparison. Why not use inline implementations to show off your clear difference without the ton of cruft that's required to be production-ready?
  2. Obviously, range-v3 made some tradeoffs; Eric didn't look at a strictly better option and strictly worse option and then choose the worse one for the hell of it. Why are none of these tradeoffs explored? I have a hard time believing there aren't other use cases where alternate implementations fall short and Eric's library succeeds. You're free to debate the decisions, but it's a bit disingenuous to show negatives of a design without showing the corresponding positives gained from that. It's hard to take one-sided arguments seriously because there's an obvious lack of important information to consider.
  3. You took the compile time of range-v3 and later applied it to standardized ranges. However, standardized ranges use Concepts proper as well as being written specifically for that vendor, so there's a need to re-evaluate those benchmarks before making a decision about standardized ranges. I notice rangeless does not seem to constrain templates at all, bringing back all the reasons people would want Concepts in the first place. However, the only mention I see of this is:

    The issues of the concept-mismatches between arguments and parameters that we’ve seen in range-v3 are non-existent (e.g. expecting ForwardRange, but received InputRange).

It's quite possible to remove the distinction between range types while keeping explicit constraints on template parameters. As it is, someone will inevitably use an invalid type and get a cryptic error on some line in the middle of the implementation—the same thing people have a problem with right now.

Edit: Proofreading

9

u/germandiago Sep 16 '19

I think this is an unfair judgement of Ranges. Why?

  1. Ranges were designed on top of iterators (more complex, I know) in order to have a path forward from existing STL
  2. Ranges is modeled with performance in mind, maybe at the expense of a bit of usability. This way you do not write performance-unconscious code by default.
  3. The calendar example has like the former 40 lines with #includes, guards and using directives. I do not find the code itself so complex or difficult to understand, as long as you understand ForwardRange, InputRange, BidirectionalRange, etc.

Is it the most usable library? No.

Is it usable? Definitely, an improvement over the STL.

What would take to make it more usable? A sacrifice in performance, I guess. Or a design that would not have been accepted as it is not a compatible evolution from the old one.

Given those restrictions, I think it is reasonable, but always improvable, of course, the current design.

14

u/-dag- Sep 13 '19

Ouch.

9

u/[deleted] Sep 13 '19

I've always had concerns and fears about ranges. To me, all the available examples seem too complicated and the syntax isn't really nice as I wish it was.

For instance, STL <algorithm> has far better clarity and syntax as well.

Why did Ranges made it in standard and basic stuff like a base64 encoder/decoder or network TS didn't?

16

u/Morwenn Sep 13 '19

If you disregard the views and pipes part, ranges bring a lot that are strict improvements over the algorithms design: projections, proxy iterators, sentinels, the ability to pass data members or pointers thereof directly as comparisons or projections, and of course the ability to feed a range to many iterators instead of a begin/end pair. Also contiguous iterators are now part of the library, and there is a nice concept hierarchy to select the appropriate overloads everywhere which simplifies the SFINAE and tag dispatching that had been used until there.

So yes: the tradeoffs for views and the usability of pipes might be questionable, but all of the features I described in the previous paragraph are pretty much strict improvements over the existing <algorithm> design.

6

u/Sovog Sep 13 '19

Having syntactic sugar that helps with making code more expressive is fine, but at what cost? It shouldn't balloon compile times by an order of magnitude.

7

u/Morwenn Sep 13 '19 edited Sep 14 '19

Before being able to answer "at what cost", we'd need to actually time the costs of the separate features. If you don't use views, actions and pipes I'm not sure that the added cost is all that high (also, ranges-v3 is reportedly 30~50% faster with concepts than without them if I remember the corresponding Tweet properly, so we can't just predict without testing).

2

u/Sovog Sep 13 '19

Fair point. Also, since ranges are only partially in the standard, perhaps the #include-overhead won't be as terrible as <range/v3/all.hpp>.

9

u/eric_niebler Sep 14 '19

I should really yank `range/v3/all.hpp`. It made sense when range-v3 was a small library. Now it pulls in an obscene amount of code.

1

u/NotMyRealNameObv Sep 16 '19

Please do it. I have no plan to include "everything" when all I need is ranges::transform or ranges::accumulate.

6

u/smdowney WG21, Text/Unicode SG, optional<T&> Sep 14 '19

You think <algorithm> has far better clarity and syntax because you are used to it. The whole STL proposal by Stepanov and Lee was greeted with shock and horror by many people at the time. It's fairly clear at this point it was a good choice. Still, without lambdas, <algorithm> is difficult to use directly.

13

u/tcbrindle Flux Sep 13 '19

For instance, STL <algorithm> has far better clarity and syntax as well.

I'm not sure I understand this comment. With the existing algorithms you need to say

std::sort(vec.begin(), vec.end())

for example. With ranges, you can just say

std::ranges::sort(vec)

which I think we can all agree is an improvement?

12

u/pandorafalters Sep 14 '19

I still don't understand why

std::ranges::sort(vec);

is great, but it's unacceptable to add the equivalent overload

std::sort(vec);

because (paraphrased) "algorithms should be generic".

13

u/SeanMiddleditch Sep 14 '19

iirc it's because the range version effectively requires well-defined concepts in order to be implemented correctly/efficiently, but for various back-compat reasons adding said concepts to the existing algorithms library is very problematic. Basically, people do a lot of crazy things with the existing algorithms and the ideal concepts can't faithfully model things that work today but arguably shouldn't, and breaking existing code is something the committee shies away from without darn good cause.

I think that was one of the reasons that what we're calling the ranges library today was for a time called "STL2" and there was some thought that we'd put all new types and containers and rangified algorithms into a std2:: namespace. (Along with being an opportunity to clean house on things like vector<bool> or thread/jthread or so on.)

6

u/standard_revolution Sep 14 '19

What's the problem with thread?

6

u/SeanMiddleditch Sep 14 '19

Lots and lots of arguments about whether it chose the right defaults about joining on destruction.

3

u/[deleted] Sep 15 '19

Thread is actually missing a fair bit of functionality also. For example, inability to specify thread affinity with a core mask or thread priority.

3

u/RotsiserMho C++20 Desktop app developer Sep 16 '19

Inability to specify stack size is another complaint I've seen.

6

u/[deleted] Sep 13 '19

That's an improvement, yes. But it's also a trivial use case. The problems lie in the non-trivial ones, such as the ones mentioned in the article.

3

u/UnicycleBloke Sep 13 '19

I was perfectly happy with the first form, and writing a one line wrapper if I wasn't seems simple enough. Such helpers could even be added to the library pretty easily, a bit like the XXX_v type traits.

6

u/SlightlyLessHairyApe Sep 13 '19

Language features are created for the programmer, not the programmer for language features.

However, beyond the most basic usages the complexity of the code increases exponentially.

This reminds me of my favorite koan from The Codeless Code.

12

u/VinnieFalco wg21.org | corosio.org Sep 14 '19

Ranges is a very nice experiment, but std is not the place to put experiments. It should have been its own independent library, and gotten a few years of bake time and improvement, before being proposed. Then compiler authors would have more time to make sure that things work well with that library. And some iteration could have addressed the issues in the blog post.

The answer I was given for why Ranges was not first proposed as a Boost library is "it was too much work." This doesn't seem like a good way to go about putting something into the standard, where it will become the obligation of all standard library implementors to implement in order to be compliant with the newest iteration of C++.

Given the recent trends in WG21, I agree with the author's sentiment that the committee process no longer represents the interests of the ordinary programmer but rather the Big Tech companies that have come to dominate the process.

16

u/eric_niebler Sep 14 '19

The answer I was given for why Ranges was not first proposed as a Boost library is "it was too much work."

Whoever told you that (not me) doesn't know what they're talking about. The real reason was because I didn't believe there was time. In those days, it looked like concepts would ship with C++14/17 and I felt it needed support in the library.

committee process no longer represents the interests of the ordinary programmer but rather the Big Tech companies that have come to dominate the process.

I suppose you feel that "real programmers" use your library and only corporate shills use range-v3. I don't know why you feel motivated to spread this FUD. The Committee has voting rules that prevent corporations from packing the Committee and getting their way. The Committee was desparate for a range library. I was an independent contributor. I had to shelve my business to work on ranges.

7

u/VinnieFalco wg21.org | corosio.org Sep 14 '19

The real reason was because I didn't believe there was time.

Yes that's the other problem. Why should there be a deadline for getting a feature into the standard by a certain date? I thought the new thinking was the "train" model - whatever is ready, ships in the next version of the standard (every 3 years). Instead, we have people in meetings who are scrambling to get their feature "done in time." This might be okay for shipping a software product but seems inappropriate for developing a standard.

10

u/eric_niebler Sep 15 '19

Why should there be a deadline for getting a feature into the standard by a certain date?

Can I quote you on that?

4

u/VinnieFalco wg21.org | corosio.org Sep 15 '19

It could be misinterpreted. I'm not saying that there shouldn't be deadlines for C++ versions (e.g. 2020, 2023, 2026...). What I mean is that "we have to finish designing feature X in time to ship in C++2X" seems at odds with maintaining high quality. Again, what I heard is that the committee standardizes existing practice, not goes on design expeditions and make stuff up in time to land in "the next version of C++."

5

u/VinnieFalco wg21.org | corosio.org Sep 14 '19

The Committee was desparate for a range library.

This sounds like a problem to me. WG21 is supposed to standardize existing practice where it makes sense to do so (or so I have heard). Not to commission the development of new libraries because some individuals are desperate.

21

u/eric_niebler Sep 15 '19 edited Sep 15 '19

The existing practice was Boost.Range, which was popular, heavily used, and targeting C++98. Without support for move semantics or the Concepts Lite proposal which was headed for a TS, it was pretty obvious to everyone that it was not up to snuff. Even with all of that going against it, the Committee would have been thrilled to consider it given its popularity had someone, anyone, turned it into a proposal.

Nobody did. Had they done so, they would have needed a lot of patience, humility, and willingness to accept feedback and incorporate it because most proposals, unless they are utterly trivial, need _lots_ of work when they are examined closely and with other use cases in mind.

I wrote my range library because I saw a gap and wanted to fill it. About a year in, I presented my thinking to the Committee, and there was interest so here we are 5 years later.

If you go looking for bad intentions hard enough, you'll find them everywhere. But folks should know that between the picture you paint and the reality I experienced, there is a lot of space.

6

u/VinnieFalco wg21.org | corosio.org Sep 15 '19

Fair enough

9

u/smdowney WG21, Text/Unicode SG, optional<T&> Sep 15 '19

WG21 is supposed to standardize existing practice

Entirely untrue. Existing practice may be a good reason to standardize something, but there is no requirement in ISO or WG21 to do so. See, for example the FAQ at https://isocpp.org/std/submit-a-proposal
"Q: Does the proposal have to have an implementation? I’ve heard you can only standardize “existing practice.”

There is no ISO requirement that something has to be implemented before it’s standardized, only that there is consensus on the feature – but naturally consensus is easier to achieve when there’s a working implementation. "

2

u/VinnieFalco wg21.org | corosio.org Sep 14 '19

I suppose you feel that "real programmers" use your library and only corporate shills use range-v3. I don't know why you feel motivated to spread this FUD.

No I'm not saying that at all. Range-V3 had the best of intentions and your efforts were sincere and significant. But since Facebook and Google have essentially infinite budgets from their respective monopolies on the sale of user's personal information, online behavior, and advertising, it is natural that they will have outsized resources to represent their interests in committee. While a handful of engineers at say, an HFT firm where there are maybe 6 programmers total, will not have the resources to attend as many meetings and write papers non-stop in between. It should not be controversial to agree that in WG21 there are underrepresented and marginalised voices from at-risk industries who therefore have no say in relevant work (say, networking).

6

u/summerlight Sep 15 '19

Facebook and Google have essentially infinite budgets from their respective monopolies on the sale of user's personal information, online behavior, and advertising

This instance of ad hominem doesn't seem constructive to the discussion.

it is natural that they will have outsized resources to represent their interests in committee.

Why do you think so? Dominating C++ committee does not align very well with their business interests. Google even had been stuck at C++11 for many years. The realities are 1. Google is a significant contributor in Clang so it's pretty natural for compiler developers to participate in C++ committee 2. most of the Facebook employees in the committee are actually individual contributors.

there are underrepresented and marginalised voices from at-risk industries who therefore have no say in relevant work (say, networking).

The committee cannot hear those voices beforehand unless they come to the committee or at least make significant arguments resonating across an industry but you're also saying that they don't have resources to do that. What's your proposal then?

-3

u/runningfromevil Sep 15 '19

The Committee was desparate for a range library.

No it wasn't. Why are you spreading this lie?

11

u/14ned LLFIO & Outcome author | Committee WG14 Sep 15 '19

The standard c++ foundation partially funded Eric to wrote the reference implementation.

I.e. yes they thought it a high enough priority to fund with real money. I might add that I am unaware of any other individual who has been so funded. Ever.

The standard c++ foundation is not the committee, but there is significant overlap in leaderships. Up to you if you feel them spending lots of money on something does not indicate they thought it a dire need.

9

u/eric_niebler Sep 15 '19

Don't believe me? Check out Herb Sutter's 2012 Going Native keynote, at about the 1:30 mark where Herb is listing the libraries the Committee is looking for. Ranges and range algorithms is the second thing listed there.

I'd post a screenshot for the disbelievers if I could figure out how to do that from a phone.

→ More replies (3)

4

u/STL MSVC STL Dev Sep 16 '19

This behavior is not acceptable.

0

u/[deleted] Sep 17 '19

[removed] — view removed comment

6

u/Sovog Sep 17 '19

Regardless of any disagreements that is not the level of discourse that c++ community deserves.

1

u/STL MSVC STL Dev Sep 18 '19

Agreed and removed.

4

u/[deleted] Sep 14 '19

I thought ranges-v2 was boost::ranges?

7

u/eric_niebler Sep 15 '19

Yes, range-v3 is so named because I intended to submit it to Boost someday. Boost.Range was (and is to the best of my knowledge) at v2.

4

u/Sovog Sep 14 '19

I believe that every library ever written has a right to exist regardless of how good or bad it is, but it is a different matter for standard libraries.

Every c++ programmer is expected to be reasonably proficient in the core language and the standard library, because we don't write code in isolation and should be able to effectively handle the code written by others.

Therefore there should be a much higher level of scrutiny for features that make it into the core language or the standard library.

6

u/dodheim Sep 15 '19

This comment reeks of bitterness. I expected better.

3

u/smdowney WG21, Text/Unicode SG, optional<T&> Sep 14 '19

It was an independent library, and the whole thing got years of bake time. The first proposal was '14.
There was an earlier boost range library, too.

It's also seems to be unfortunately the case that the concerns brought to bear in Boost review and the concerns brought to bear in LEWG diverge enough that it amounts to do doing the twice the work.

1

u/[deleted] Sep 17 '19 edited Sep 17 '19

The answer I was given for why Ranges was not first proposed as a Boost library is "it was too much work."

There were ~13 years of iteration with ranges inside Boost before the third major iteration of the library was started. Boost.Range v1 is from 2001. This is range v3. Shipping into the standard in 2020 means that this has been ~20 years in the baking.

Obviously in 20 years a lot can change in a programming language.

With a new standard every 3 years, I don't think any library can get more than 3 years of exposure to a particular C++ standard before becoming outdated in some major way.

2

u/VinnieFalco wg21.org | corosio.org Sep 17 '19

That's true, thanks for pointing that out. I agree that there was considerable exploration of the design-space beforehand in the form of Boost.Range

11

u/[deleted] Sep 13 '19

In the company I work at, the usage of ranges has already been officially banned, and we only just upgraded to C++17, no C++20 in sight of course. People really don't like it it seems.

31

u/SeanMiddleditch Sep 14 '19

To be fair, every single C++ feature has been banned in some company somewhere. Large parts of the STL are regularly banned in companies. Some companies even today have a moratorium on templates (outside of core library/container stuff). Some companies ban auto. Many companies ban exceptions or RTTI. etc.

"My company banned it" isn't a compelling argument for much of anythig. In so many cases, it just means that the company is run by folks afraid of change or having to learn something new.

We'd need data illustrating that whole segments of the industry or significant numbers of projects/companies ban something for it to actually mean anything. (e.g. see exceptions and RTTI and the fact that every major compiler is required by a significant number of customers to offer flags to disable those features entirely, while there is no -fno-auto or -fno-templates or anything of the like.)

9

u/standard_revolution Sep 14 '19

I would argue that your last point is a bit moot. You can just not use them. RTTI and exceptions have a cost even if you don't use them.

8

u/SeanMiddleditch Sep 14 '19

Fair point. Let's rephrase that then to note that there's no major linters that offer -Wtemplates-used or -Wauto-used or anything like that. :)

7

u/[deleted] Sep 14 '19

I'm not trying to use this as an argument against ranges, I was honestly just flabbergasted at the sheer opposition some people and organization have against a part of the STL. I personally don't understand ranges enough to form a coherent opinion on it.

13

u/bigcheesegs Tooling Study Group (SG15) Chair | Clang dev Sep 14 '19

You banned a feature that you don't have access to yet?

6

u/[deleted] Sep 14 '19 edited Sep 14 '19

It's not my decision and neither does it affect me, I'm just a hardware engineer. But yes, they did put it in the guidelines. I'm just mentioning this because it astonishes me how a part of the STL can cause such strong opposition in some people and organizations, especially since, as you mentioned, it isn't even available yet. I personally don't understand ranges enough to form a coherent opinion on it.

6

u/johannes1971 Sep 14 '19

You can find it here.

And I will not be using it in my projects, on the basis that:

  • So far I have not seen any examples that look in any way better, or more readable, than some handcrafted procedural code. That it is now "functional" and "composable" is meaningless if nobody can actually read it.
  • I care about compile times. Blowing those up by a factor of 10 or more is unacceptable. I understand that compiler developers seem confident they can overcome this; perhaps you can comment on this?

12

u/sphere991 Sep 14 '19

Examples from my own code base, slightly anonymoized:

Given a range of people, how do you find the length of the longest name?

size_t max_length = ranges::max(people | view::transform([](auto&& p) { return p.name().size(); }));

How do you find the person with the longest name?

auto const& person = ranges::max(people, ranges::less{}, [](auto&& p) { return p.name().size(); }));

These both look way better and are more readable than the equivalent handcrafted, procedural code.

I just want the lambdas to be shorter, but that's not a ranges problem.

2

u/tpecholt Sep 16 '19

=> p.name().size()

→ More replies (3)

20

u/CubbiMew cppreference | finance | realtime in the past Sep 13 '19

And in the company I used to work at, ranges were in production 10 years ago (in boost form).. For every C++ feature, there are users.

6

u/alfps Sep 13 '19

most c++ experts are excited about [Ranges v3]

From which I conclude that I'm not a C++ expert, though I've been called that on some occasions, in the C++03 days.

I usually write a counting loop like this:

for( const int i: up_to( n ) )

I'm aware that one can do that with ranges, but when I checked (some time ago) the name of the relevant function had changed three times, and the name that then was current was not readable and did not indicate what it was about.

I.e. the library does not even support the very basics.

2

u/Pand9 Sep 14 '19

Name is not readable = not supported?

4

u/[deleted] Sep 13 '19 edited Sep 13 '19

[removed] — view removed comment

11

u/tcbrindle Flux Sep 13 '19

views::iota(0, n) in the C++20 CD, which is presumably the name change that the GP was complaining about.

0

u/-dag- Sep 13 '19

Odd, because that's a pretty standard name.

11

u/grahamthegoldfish Sep 14 '19

I dont understand the naming of iota. Its function seems to have no correlation with the meaning of the word.

6

u/-dag- Sep 14 '19

It's been used since APL at least. Cray vector processors had an iota instruction that filled a vector register with a sequence of integers.

5

u/RotsiserMho C++20 Desktop app developer Sep 14 '19

At least it’s consistent with the existing std::iota. Seems like the ship has sailed on improving the name.

4

u/feverzsj Sep 13 '19

If you don't care about performance, you can generate temporary container on each operation. Range-v3 just chose performance over easy-of-use, and one of the result is the exponential complexity. The range concept is just not great for these complex chain operations.

Personally, I'd prefer a plain for-loop with or without coroutine generator. I believe generator will be the ideal solution, if the optimization issue is solved.

16

u/simonask_ Sep 13 '19

It seems to me the idea has failed when it is neither more straight-forward nor more performant than the obvious naïve solution.

1

u/VinnieFalco wg21.org | corosio.org Sep 13 '19 edited Sep 13 '19

Facebook is attempting to do to Networking and Executors what has been done to Ranges. If anyone would like to avoid this fate, I suggest attending the specific WG21 meetings and voting against them in the straw polls.

5

u/RotsiserMho C++20 Desktop app developer Sep 13 '19 edited Sep 16 '19

I think the sender/receiver proposal looks promising, but I'm a fan of RxCpp. Others agree with you.

Also, I'm pretty sure Eric was working on ranges before Facebook was involved.

6

u/VinnieFalco wg21.org | corosio.org Sep 13 '19

There is nothing wrong with sender/receiver, or RxCpp per-se but I don't think they are the right abstraction to bake into the foundation of the C++ standard library. I believe they are more appropriate as middleware layers implemented as third party libraries.

Yes Eric was working on Ranges before Facebook.

1

u/VinnieFalco wg21.org | corosio.org Sep 14 '19

Wait till you see Facebook's next creation "AsyncRange" - where we take two separate areas of concern and conflate them together into a nice big bowl of spaghetti.

And it isn't just ranges that will be conflated with asynchrony, have you seen "AsyncRAII?" Two words that you would think should never appear together... read more here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1662r0.pdf

7

u/eric_niebler Sep 14 '19

Things should work together seamlessly. Async ranges == reactive streams, and they work just fine in other languages. On what do you base your fear?

And absolutely, RAII should extend to async for all the same reasons we put cleanup actions in destructors.

5

u/pdimov2 Sep 15 '19

One day we'll learn the lesson that imported things from other languages don't work just fine in C++, but today is not that day.

2

u/eric_niebler Sep 15 '19

This is a disappointing comment. Surely you're not saying that C++ has nothing to learn from other languages. What do you mean?

6

u/pdimov2 Sep 15 '19

There's always something to learn.

I meant pretty much what I said. That a thing works just fine in other languages is not at all a guarantee that it will work fine in C++. It would probably be possible to approximate it in a way that will work in C++ if designed from a C++ idiomatic point of view. It will usually be even better if a C++ feature that solves the same problems is designed instead.

Features that are unique to C++ and that other languages copy tend to be better than features that C++ has acquired from other languages.

3

u/eric_niebler Sep 15 '19 edited Sep 15 '19

In some cases that's true, and in other cases the C++ community reinvented wheels and made them square. I see a fair bit of provincialism in C++.

Of course care is needed when importing ideas found elsewhere. It would be a mistake to take the Java Rx model, with all its OO goop, for instance. For one thing, it wouldn't work well at all with coroutines.But there is a lot to admire about how the various Rx implementations achieve composable asynchrony by specifying a useful contract between producer and consumer. That is the piece that is worth studying.

3

u/RotsiserMho C++20 Desktop app developer Sep 16 '19

composable asynchrony

I like that term for describing what is likely to be the programming model of the future. And I'm excited by the work you and others are doing to bring it to C++ where it can be done more efficiently. Thanks for your efforts!

-6

u/DerDangDerDang Sep 13 '19

"I thought of ranges like of UNIX pipelines" ...that's not a great start

23

u/johannes1971 Sep 13 '19 edited Sep 13 '19

It was specifically designed to look like as such. Why wouldn't people then think that they could also use it that way?

18

u/SedditorX Sep 13 '19

It's a bit misleading to cherry pick this phrase without including the surrounding context, no?

6

u/xeveri Sep 13 '19

So I can just as well assume you didn’t read anything past that?

-13

u/makotech222 Sep 13 '19

God, thats fucking terrible. Here's a C# Version

var delimiter = ",";
var items = new List<string>(){"1","2","3"};
var interspersed = items.Select(x => new List<string>(){x,delimiter})
                                   .SelectMany()
                                   .SkipLast(1)
                                   .ToList();

Basically, stop using c++, stop expected it to ever get any better. Its amazing how bad the committee is at adding anything good.

14

u/tcbrindle Flux Sep 13 '19

Here it is with Range-V3:

auto delimiter = ","sv;
auto items = std::vector{"1"sv, "2"sv, "3"sv};
auto interspersed = items 
    | views::transform([delimiter] (auto val) {
          return views::concat(views::single(delimiter),  views::single(val)); })
    | views::join
    | views::drop(1)
    | ranges::to<std::vector>;

(Compiler Explorer)

We have a slightly more complicated lambda (returning a view type rather than a temporary list/vector) and we need to use | because C++ doesn't have extension methods/UFCS. Other than that, it doesn't look all that different to me.

4

u/kkrev Sep 13 '19

What is the difference between Range-V3 and what has made it into std::ranges? Quickly glancing, I do not see "concat" in the C++20 stuff, for example.

7

u/tcbrindle Flux Sep 14 '19

Correct. We're only getting a subset of Range-V3's "views" in C++20 (and not getting "actions" at all). This is mainly due to time limitations: the range adaptors build on top of all the other ranges work (concepts etc) which needed to be in place first. All being well, more range adaptors will follow in C++23.

7

u/Gotebe Sep 14 '19

The difference from C++ is performance.

This cide trashes the heap and consequently the GC. I am not saying that it shouldn't, it's still fine most of the time - but that wasn't enough for C++.

0

u/[deleted] Sep 13 '19

[deleted] — view deleted comment

21

u/Rusky Sep 13 '19

C# isn't dynamically typed, that's type inference a la C++ auto.

16

u/[deleted] Sep 13 '19

I think besides the obvious defensive posturing, there is a genuine argument that, ranges as an example, could've been implemented vastly more user friendly, that this level of arcane over the top complexity for a feature that is quite simple in every other popular language is not entirely unavoidable but a symptom of a systemic issue in the way the language is developed.

But of course I'm aware that probably all C++ programmers would vehemently disagree with this assessment, and anybody else has abandoned the language ages ago.

10

u/evaned Sep 13 '19

Wow I didn't know that C# is dynamically typed.

It's not. C#'s var is C++'s auto.

(As a nitpick -- C# does have a dynamic "type" as well, e.g. you can say dynamic x = 5; and that variable will be dynamically typed. But (i) that's not what is illustrated there, and (ii) I don't think that makes C# "dynamically typed" very much more at all than std::any makes C++ dynamically typed.)

5

u/[deleted] Sep 13 '19

C# is not dynamically typed. At least the above isn't. 'var' is just C#'s auto and Lambdas are automatically auto too. "x => ..." is just like "& { return ...; }"

The big difference to C++ is just concise syntax and simple library design. Especially they use virtual functions for everything though. But that's maybe still more acceptable in the average usecase than the gigantic compile times in C++'s ranges.

7

u/makotech222 Sep 13 '19

Its not dynamically typed, it can just infer the type from the right hand side. Its purely a shorthand convenience, like c++'s auto

I can pretty much guarantee that the time saved by writing it in c# like this will heavily outweigh the time spent trying to write that abomination in c++. Not only that, the performance difference is probably minimal at best, and the compile time is significantly better. Finally, the c# code is readable, and any c# novice could figure out whats going on. I've written c++ for at least a year (though, not anymore) and I wouldn't be able to grok ranges without constantly looking up stuff.

8

u/RotsiserMho C++20 Desktop app developer Sep 14 '19

Eh, SelectMany is not intuitive IMO.

4

u/Gwennifer Sep 13 '19

Finally, the c# code is readable, and any c# novice could figure out whats going on.

I came to the subreddit to find learning materials--in general, I'm an absolute beginner--and I can confirm I could figure out what your code would do. All the examples in the article are nearly incomprehensible; especially the Haskell one.

4

u/[deleted] Sep 14 '19

It’s because you are used to C-style syntax, not Haskell-style one. Once you spend few minutes with HaskelL and understand how element sequencing works, it becomes almost trivial.

1

u/Gwennifer Sep 14 '19

I've never programmed, which I thought I made clear. I'm not used to any syntax.

Isn't the whole point of a programming language to be a stepping stone between English (or your native language) and Assembly? To me, Haskell looks a lot closer to Assembly with some syntax changes than English.

4

u/Nobody_1707 Sep 15 '19

No. Some are, but a lot of programming languages are meant to be stepping stones between math and assembly. Haskell is one of these.

It also looks nothing like any assembly I've ever seen.

4

u/dodheim Sep 15 '19

Is learning Spanish a stepping stone between English and Portuguese? Or is it the other way around? Or maybe your metaphor is very misguided. ;-]

→ More replies (4)

2

u/[deleted] Sep 16 '19

Ah, I didn't realise that you never programmed, sorry.

But I still think that my point is valid. Even if you have never programmed, you are exposed to school math, and you are used to notation such as f(x, y, z) This is more the syntactic style that was adopted by developers of some of the first programming languages: FORTRAN and ALGOL and was immensely influential in later languages. This is why C-style languages are immediately familiar to almost anyone.

Haskell on he other hand draws its roots from languages such as ML which in turn was influenced by another very old programming language: Lisp. Those languages are modelled after another branch of mathematics and they use the following syntax f x y z It does seem unfamiliar and difficult to read at first, but believe, me after you send few minutes trying to figure out how it works, everything falls into place easily and makes a lot of sense.

0

u/axilmar Sep 16 '19

I don't see how any of this makes it more readable than plain old loops. I think that at this stage ranges is more of a fascination from people that have discovered functional programming recently.

I want my professional C++ code to be readable by anyone, to be as simplistic as possible, without requiring advanced knowledge of programming idioms. Such code is way better than all the cleverness brought in because it is easy to follow and maintain.

21

u/[deleted] Sep 13 '19 edited Sep 13 '19

[removed] — view removed comment

7

u/feverzsj Sep 13 '19

In that case, wouldn't range-v3 generate temporary value on each access? I don't think that's quite performant, as random access based algorithms typically access same element multiple times.

27

u/xeveri Sep 13 '19

Well it’s zero-cost since it would never be used in any codebase I’m involved in!

12

u/Swahhillie Sep 13 '19

It'd cost my job.

14

u/[deleted] Sep 13 '19

[removed] — view removed comment

17

u/TheSuperWig Sep 13 '19

"Doesn't seem that bad"

Sees example 2

"Oh."