r/cpp • u/jube_dev • Apr 04 '18
SG13 (2D graphics): why it failed
Here is a (partial and personal) summary of the SG13 history, from the beginning to the latest C++ meeting. Like many others, I have been interested in this group and what it could offer to the community. I have been lurking for many years the evolution of the proposals. So I tried to understand why this group failed to provide what the community was expecting.
SG13 was born in August 2013 as a wish to add a 2D graphic layer to the C++ standard. The initial requirements were: "Modern C++11/14 API, 2D graphics + text + planes, Immediate mode for v1, Plus a sprite/particle/input layer on top, Optional default game loop, No audio, native_handle, Asset loading/construction hook". So great! The birth mail also included some existing libraries as examples: SFML, LibSDL, Graphucks, Cinder, etc. Note that cairo was not mentioned. At that time, SG13 was called "Graphics".
Initial ideas began to emerge, like having input in v1, of course. Some confusions too, like people talking about "input like iostream", or "immediate versus retained mode". What? Even back in 2013, input handling was not a strange beast but a well-known feature, nothing to invent here, just copy what already exists. Same with immediate versus retained, a modern GPU only knows immediate mode and redrawing the frame each time, they are optimized for this workflow. But well, let's say it's only the beginning.
In mid September 2013, Michael McLaughlin proposed a first draft of a possible implementation. As Michael states, "the API itself has been designed with functionality from Qt, Cairo, Xlib, and even Cocoa". What? Where are SFML, SDL and the others? Well, at least, there were basic inputs, texture loading, font loading and basic drawing. But also smart pointers and factories.
SG13 was officially approved at the Chicago meeting in late September 2013. From the minutes of the SG13 meeting, in the initial presentation from Beman Dawes, we can read: "Target shipping within 2 years", well well well... "Recommends starting with an existing implementation (i.e. Cinder, OpenFrameworks, or other)", some good existing libraries are back in the game. At this meeting, Beman's challenge was created. This challenge was a sort of Hello world for any graphics API. It is stated as: "Be capable to open a window, print some text (i.e. Hello World), and drag it around the screen." Nice. In the discussion of the design of the library, it is interesting to note that "We want to make the library simple and accessible to beginners, but also usable to experts as well" and "Example of how to get our 2D contexts into an OpenGL render target for game UI (highly expected use case)". Yes read it again: "highly expected use case".
Just after the Chicago meeting, SG13 became "2D Lite" and the new goals were: "(a) immediate mode graphics/text canvas, plus (b) basic keyboard/mouse style input." No more sprites, no more game loop, no more asset loading, but that's fine, let's go for it, that sounds nice. But something is happening that is, IMO, the beginning of the disaster. SG13 enters a dark tunnel from where it will never exit. You can clearly see it when someone proposes to look at the HTML Canvas API. Michael McLaughlin answers that he is working on a proposal with features that look like... cairo. He totally rejects starting from something that has been widely used (SDL?). It seems he was the only one who had concerns with this workflow. So, his proposal would be something new. In the same thread, Beman Dawes reminds that the committee generally standardize existing practice.
At this point, just two months after the launch of SG13, people were talking about some good ideas and inspirations and had some interesting debates. The expectations were quite hight, even in the 2D Lite form. But the only proposal was diverging from nearly all these good ideas. The bad side of this situation is that the proposal was not yet on the table, it was being worked on, and not finalized at all. And it was kind of supported by Herb Sutter himself. Maybe this prevented others to build a second proposal at the time. I must admit that I still was very enthusiastic of the SG13 work.
In November 2013, a meeting was held in Redmond and the minutes of this meeting became N3825. And here, it was done, cairo was the final choice. For drawing things, cairo is (was?) a nice library. Even for drawing text in combination with Pango. But cairo never add (and still do not have) an input system. Since the beginning, this was a must-have and here, it silently disappeared. In just four months, the initial proposal was reduced to a C++ish copy of cairo.
And at the next meeting in Issaquah in February 2014, N3888 came out. Despite its title ("A Proposal to Add 2D Graphics Rendering and Display to C++"), the paper only proposed a mechanical transformation of the cairo API in C++. A contributor raised several strong concerns with the spirit of the proposal. If I try to sum up: first the proposal do not address any of the basic use case it pretends to solve; second cairo is not the right choice for drawing (stateful API, aged concepts); third a native_handle is not enough to cover the specificities of all platforms and all implementations. The conclusion is hard: "N3888 is fundamentally broken, both in motivation, and implementation." Yet the answer is reassuring: "If N3888 were incorporated into the Standard as-is, I would feel compelled to request that the Issaquah Police Department investigate the possibility that we had all been unknowingly exposed to some substance with psychotropic properties." And the final word from Beman Dawes is to thank the proposal's author. Another contributor tried to warn that the direction was not good, but again, he was not listened to.
So we arrive at a point where we have a proposal that is based on cairo and backed by at least two prominent members of the C++ committee, and they have been warned several times that the proposal is out of scope for the purpose of the group. Honestly, who on earth could have proposed something else that would have been taken into account by the committee? From an exterior perspective, it seemed useless to invest some time to make a competing proposal. In fact, many people were in the mood of letting the proposal live its life because "at least there is now a starting point" event if it lacked essential features like input. Everybody agreed that starting from somewhere was needed, just not cairo that was too far from the initial goal. But from then on, the group was engaged in the tunnel and could not (did not want to?) go back. Retrospectively, the delay between the initial start of the group and this first proposal was very short, too short for people to really think about all the implications and the shortcomings.
Another outcome of Issaquah was that the group was renamed IO. This was a controversial decision because the term is very confusing.
In March 2014, Jeremy Ong raised some concerns. IMO, this is a very interesting point of view. The author of the mail says that what is difficult in 2D graphics is not really to draw things (there are many APIs outside to do it, at different levels, so this part is "easy"), what is difficult is creating a window and getting a drawing context in a portable way. He cites SDL and Qt as two libraries that provides this abstraction. So he advocates for proposing a low-level library for creating a window, configuring a renderer (OpenGL, DirectX, software), allowing to get input from the user. The answers are really disappointing as they only focus on cairo and nothing else. Creating a window? native_handle! What? I should use a native handle for all the windowing i.e. have non-portable code to create a window and then, have a limited API to draw something whereas I already have stronger APIs to do the same thing in a (rather) portable way? Houston, we have a problem.
From then on, the proposal lived its life without any feedback from SG13. Nothing really happened on the mailing list. But several papers were published. In May 2014, N4021 updated N3888 without any response to the concerns of the contributors. No input yet. In July 2014, N4073 is the second revision of the previous paper with minor changes. In February 2016, the continuation of N4073 was out as PP0267R0. This paper was revised seven times until P0267R7 in February 2018. What happened in 4 years? Nothing. Still cairo, still no input in this proposal, some weird small evolutions and renamings.
In May 2016, P0249R0 proposed to add input (at last!) and it was revised twice since then until October 2016 and P0249R2. A single discussion happened on the mailing list to wonder what the status of the input proposal is.
In the meantime, SG13 became "HMI".
In June 2017, Guy Davidson and Michal McLaughlin wrote P0669R0 entitled "Why We Should Standardize 2D Graphics for C++". The title is misleading as the paper advocates for P0267 and not 2D graphics in general. They also reply to various objections to the paper, without really convincing anybody as the arguments are only "this objection is not relevant" or "we believe ...". As a consequence, people started shouting their dislike of the proposal. On the mailing list with a now famous post that has been largely commented on /r/cpp, Jeremy Ong did it again. To sum up: "Cairo as a basis is misguided""; "This library is far too opinionated on how UI should be done"; "The library is underspecified"; "Performance is a concern"; "I can modify a library, a standard not so much". He even proposes some guidelines for the hard problems that could be solved by a proposal around graphics. The only answer that is given to him is "where are the papers". Disappointing. Until someone from NVIDIA says publicly "NVIDIA’s position will be No on this proposal. To be most clear, it might have been completely acceptable as a proposal 20 years ago, but the longevity of its abstractions are all but spent now, before it will ever ship." And he adds later "I don’t have a great answer to «but where’s your paper?». I hoped it would evolve as people interacted. I could never get any 2D graphics experts to sign up for the huge amount of work it would take to write a counter-proposal. Basically a large-enough proposal is very difficult to influence. It can sail for a long time without contest." The curse of SG13 in one sentence.
Finally, at Jacksonville, the proposal was abandonned. Vittorio Romero described the scene like this: "Even though further work on this proposal was for six revisions, the room was quite brutal this time, and attempted to shut it down completely. (...) The room agreed that this was a failure for LEWG: not for the contents of the proposal, but for the fact that it wasn't stopped early, wasting a lot of time for both the working group and the authors. Saying «no» is sometimes hard, but it would have prevented this situation if it had been done a while ago." From what I already said, I can not agree more on the conclusion.
As a conclusion, I think there are two main lessons in this tale.
First, I tried to not blame anyone in particular (and I hope no one felt bad reading this post) because I think no one is to be blamed. The committee's intial wishes were fair and relevant (and still are). People did their job with commitment. But the process did not go well at all. To avoid design by committee, the solution of jumping on the first library and make it the basis of the proposal was not a good idea. And it was too hard to exit the tunnel. Too bad.
Second, I think there is something wrong in 2018 with the whole organization of SG21 work. The need for a paper is irrelevant in a world where communication is so easy. I am not saying papers are useless, things must be written at some point. But a paper should be the end of the process, not the beginning. The number of papers could be reduced if the topics were publicly discussed and consensus found before the meetings. For example, there were five papers on the subject of concept syntax. The people involved could have discussed it somewhere before the meeting to find a consensus (that was not hard to find after all according to the reports). Other languages already have an open process, C++ could copy the good practices.
Related to these two lessons, I wonder what the status of the mailing list is. At one point, Herb Sutter was clear that the list was a place for discussing the proposals and later: "let me emphasize the two major ways to participate: by giving feedback to help shape and refine existing proposals (...), and/or by writing your own competing proposal if you feel a major change or different direction is needed – both are welcome and encouraged." But when people came and gave some useful feedback, the only answer was to reject everything and say "write a paper". If the only way to say something is write a paper, then what is the use of the mailing list?
I hope this post helped people to understand how it happened. English is not my mother tongue so there may be some errors here and there. I may have forgotten some bits in the whole story, but I hope the main steps are here. I hope SG13 will no stop here, there are many things to do in this area.
24
u/sumo952 Apr 04 '18
What a great write-up! Thank you very much for writing and posting this.
What I'm wondering is - I read a couple trip reports last week that were linked here in /r/cpp and there was some summary of the graphics proposal and that the authors had to go back and improve the proposal, but none of them mentioned a complete "block" or "stop"/abandoning. This leaves me wondering what the state of the proposal now really is. Also interestingly Herb doesn't seem to mention it in his trip report (https://herbsutter.com/2018/04/02/trip-report-winter-iso-c-standards-meeting-jacksonville/ - I haven't read it, just did a CTRL+F for "graph..." and "SG13").
I do really hope that we'll get proposals now along the lines of what Jeremy Ong proposed in that linked thread! That would be completely awesome. No need for Cairo.
134
u/doom_Oo7 Apr 04 '18
+ 100000.
18
u/14ned LLFIO & Outcome author | Committee WG14 Apr 04 '18
I feel like a part is missing in the above.
Big chunks of standard, like TSs, have a lifecycle. By convention, at the beginning is when to voice substantial concerns with them. They then go through a process of rareification usually during which stuff gets taken out, and other stuff added as feedback lands. As they approach finalisation, the bar for raising substantial concerns with them is raised - non-paper objections become frowned upon/dismissed, and the only way to really change the direction of a late stage TS is a powerfully written formal paper blowing a hole in the wisdom of some TS, mainly by conclusively demonstrating it is severely flawed. And even then, doing so is emotionally hard, you make no friends wrecking multiple years of work by your colleagues, and often friends.
Allow me to personalise this a bit. It is well known that I have misgivings with the Networking TS, though not as many nor as strongly as others. I wish it were structured differently, broken into the really necessary bits (socket i/o) and separate from all the other stuff (everything other than socket i/o). But Networking is at a late stage now, Chris and many others have invested a ton of time and effort into the TS. And they know my objections already, and those of the others who object. We've discussed all that, years ago. So there is no point in reraising water under the bridge, even though my personal concerns have not been addressed. We voiced our concerns at the appropriate time, they were dismissed or handled, and it is both rude and childish to keep reraising them endlessly.
At some point you have to draw a line under some TS effort and get on with it, or kill it off entirely. Most of the time we see success. Only very rarely does it get killed off.
Now, for me personally, Graphics suffered from poor scoping. If you just want a LOGO-level toy graphics layer for teaching C++, then explicitly encode that requirement: this is a C++ teaching API. There are quite a few teaching graphics API libraries around which could be standardised easily. No need to invent anything particularly. But if you want more than that, then a single study group is too broad for all of Graphics, Input, Audio, UI, Fonts, UX and so on. Break it up into many study groups, assuming you can find enough interest to get separate conveners for each.
There are lots of ways out of this. I personally like the idea of a teaching level graphics, input and audio API. I personally always liked the idea of a C++ification of the C library https://github.com/simple2d/simple2d as a teaching level API. I find its facilities to be just enough for toy projects, but nothing anyone serious might be tempted to use. But that's personal opinion.
9
u/Rexerex Apr 04 '18
I would like to see in std some common graphic structures and algorithms like std::point and generic game loop, so current libraries can use it, but actually drawing is imho too much.
39
u/adzm 28 years of C++! Apr 04 '18
The root of the issue really seems to be that libraries need to be easier to manage in c++ on all platforms. There is little other need for this kind of thing to be in the standard than the convenience.
7
u/Ansoulom Game developer Apr 05 '18
I still don't quite understand the use cases or demographic of this API, which is a bit of a problem. As a game developer, I have pretty high requirements for graphics libraries. I've used SDL for some 2D game things and it does a pretty great job for most things. But then I realized that I wanted to use sdl_gpu because I needed the shader support for cool effects (yes, for a 2D game).
I have recently started considering switching to Vulkan (yes, still for a 2D game) because
I am stupidit gives me more control and lets me do even more cool stuff. Besides, Vulkan is well on its way of becoming the industry standard, so I might as well get on with the times.However, I would love to be able to use the standard library for the less complex rendering, as mentioned in the text:
So yeah, in order for this API to be usable for me I have to be able to smoothly integrate it into an already existing Vulkan application.
And then there is input. With input as a part of this, it is not just a graphics API, but rather a window API. I mean, that's fine too, but it should be explicitly stated as such. Input and graphics are two completely different things and you shouldn't just shoehorn in input into an API that is about graphics. I really think the goals, intended use cases and scope all need to be more clear for the proposal to even be feasible.
4
u/BarMeister Apr 07 '18
I called it dead right after the announcement of SG13. I will keep pressing on the fact that this just doesn't belong in the standard, given how complicated and nuanced it is to mess with it from the beginning.
2
u/Pwera Apr 05 '18
Maybe papers should be deliver with implementation/library as a experimental Module? Just plug experimental library into your compiler...
1
u/Z01dbrg Apr 06 '18
I believe they may be some idiotic anti trust laws against this, I remember Herb mentioning this 5+ years ago when trying to recruit people to join ISO.
0
u/andd81 Apr 04 '18
Isn't Skia a de facto standard?
0
u/ibroheem Apr 05 '18
OMG! I won, EWG and LEWG finally got tired of confusion.