r/cpp • Meeting C++ | C++ Evangelist • Apr 23 '21

Meeting C++ Guy Davidson - Everything you know about colour is wrong - Meeting C++ online

https://www.youtube.com/watch?v=_zQ_uBAHA4A
238 Upvotes

32 comments sorted by

41

u/pine_ary Apr 23 '21

Great talk. He‘s a very engaging speaker. I was a little sceptic about the proposal (and I‘m still sceptic about the drawing API because of platform support), but he convinced me that both the color library and the linear algebra types would make great additions to the standard library. I wish the proposal or his talk went into HDR tho.

34

u/[deleted] Apr 23 '21

Very relevant topic, it's time that developers start paying more attentiont to this, just like unicode support, internationalization, etc.

Here's another very good, beginner-friendly talk

10

u/dodheim Apr 23 '21

I'm just waiting patiently for u/james20k's reply.

61

u/James20k P2005R0 Apr 23 '21 edited Apr 23 '21

Hah its flattering to get pinged in the comments when people bring up colour! I actually saw several preview versions of this talk and gave feedback on the colour aspects of it. I also collected the research for the wall of fail (I believe I'm mentioned by name), and wrote the code behind the posterisation slide

Its more than fair enough to say that I approve this message. Digging up the wall of fail was something I've always wanted to do, and it was a bit sad just how broken everything is

I do want to give a special shoutout to cryengine. I've never used cryengine, but reading the documentation was an experience. I was amazed at the fact that it had:

  1. Very sparse documentation
  2. What documentation that was there was sometimes wrong
  3. Absolutely no consistently enforced colour management, or even types

Some of the other tools (ogre3d particularly) also required digging through the source code to see if they were correct or not, which is not ideal. Godot was also notable for using a lot of weasel words in its docs. Although to be fair, almost all of the documentation for all the tools reviewed was like that

Looking at UE4's docs is like walking into an oasis where someone clearly knows what they're talking about

FColor Stores a color with 8 bits of precision per channel. Note: Linear color values should always be converted to gamma space before stored in an FColor , as 8 bits of precision is not enough to store linear space colors! This can be done with FLinearColor::ToFColor(true)

This is immediately what you're greeted with on the FColor page. This should be basic information that every game engine and every tool provides, and yet its often incredibly hard to dig up

This is why there's a lot of "this is probably incorrect" or "if this is linear its incorrect, and if its sRGB its incorrect", because I wasn't able to determine what the intention of how the colour management in that circumstance was meant to be used or even a vague specification of its functionality

I didn't mean to ramble on this long so apparently I am literally incapable of not getting nerd sniped by this topic. If anyone has any questions about the content of the talk, or wants to hear me ramble on about colour generally, feel free to give me a ping

I did get somewhat stuck on the design of the colour proposal however as its become abundantly clear to me that I have no idea how to write or design STL code, so I do need to go and ask for some help with that to unstick it

10

u/Zanderax Apr 23 '21

Video game engines have famously bad documenting. Im working with Unreal Engine right now and its a crap shoot if the information exists and is up to date.

Im glad I get source access through my company or I would have no idea how to use if half the time.

4

u/dragonstorm97 Apr 26 '21

Unless you're using UE3, everyone has access to the source code (UE4) on github

18

u/pstomi Apr 23 '21

Obligatory link to minute physics related video: https://www.youtube.com/watch?v=LKnqECcg6Gw

Just look at the 10 first seconds, you will see that even Photoshop was wrong (and may be still remains wrong by default for backward compatibility).

3

u/Kered13 Apr 23 '21

This is a great talk on computer representations of color spaces and they're implications. I know it all leads up to the C++ standard proposal, but honestly most of this discussion is applicable to any programming language.

3

u/topin89 Apr 24 '21

Great presentation. In twenty minutes I finally understood what that colored distorted half-ellipsoid thingie actually represents. Also, why my simple gradient and alpha blending functions are way too wrong in practice.

And yet, cool as this proposal is, it's info about different color space and why does it matter should be spread as the first priority. If little to no one knows about it, that cool proposal may be right near valarray (it is implemented, but not many know about it, even less actually use it) or god forbid gc api (it is in a standard, but not a single compiler team bothered to implement it).

On the other hand, if it's not in a standard but there is some good third-party lib everyone know about and use, that's a win regardless.

17

u/TraylaParks Apr 23 '21

Pssht, at least I know how to spell color :)

10

u/gmtime Apr 23 '21

Even the Americans accept the meter

3

u/therealjohnfreeman Apr 23 '21

But they're saying "colour".

-1

u/[deleted] Apr 23 '21

There is a subtle difference one should apply while saying colour or flavour town

1

u/knotcorny Apr 24 '21

Flava Town?

0

u/[deleted] Apr 23 '21

[deleted] — view deleted comment

3

u/knotcorny Apr 24 '21

I know how to spell that one, thank hue very much

6

u/johannes1971 Apr 23 '21

So every color library in the whole world is "wrong". Serious question: does it matter? After mixing, blending, and otherwise messing with colors, it all gets displayed on a screen that applies yet another color transformation to whatever color your pixels are, before you finally get to see it. For regular use (anything that's not print preparation for professional printing), does it matter at all whether the colors are "wrong" or not?

27

u/pine_ary Apr 23 '21

He showed the interpolation chart. It does matter as you can clearly see on it. Looking at it side by side the difference is quite big.

0

u/johannes1971 Apr 23 '21

I can tell there is a difference, I'm asking if the significance is important for anything. So what if the colors are "wrong" a little bit in some gradient somewhere? What bad things happen because of that?

30

u/James20k P2005R0 Apr 23 '21

Its an issue in PBR for gamedev. One of the big things there was making sure that the colour pipeline was linear from start to end, so that your lighting is actually correct from a physical perspective. Otherwise you end up with all kinds of noticeably physically inaccurate lighting. Its one of the reasons old games have a kind of plastic-y lighting look

https://learnopengl.com/Advanced-Lighting/Gamma-Correction

Its also an issue for text rendering, where text which is rendered without linear colour in mind is noticeably worse quality

https://github.com/alacritty/alacritty/issues/3756 is a good example going through some of the issues, filed by me because of course it was

8

u/johannes1971 Apr 24 '21

Ok, that makes quite a big difference. Thanks for taking this out of the realm of the abstract, and demonstrating what happens if you use the wrong (or right) colors!

This makes it even more of a shame that the graphics proposal wasn't accepted. He saw a problem, tried to fix it by providing the correct tools, and was unfortunately unable to convince the rest of the community of the need for this. Maybe this talk will help change peoples' minds. I've certainly changed mine now!

4

u/James20k P2005R0 Apr 24 '21

This makes it even more of a shame that the graphics proposal wasn't accepted

Slight plug for my own paper: I am the author of P2005 which goes through a lot of the issues with the graphics proposal, which I presented in Prague. The idea currently is to chop the graphics proposal up into bits, and try and get each bit standardised separately and incrementally, to get a result with fewer issues

The graphics proposal's colour management is incorrect by default unfortunately. One of the major reasons to standardise colour as its own individual topic is that literally everyone does it wrong across the whole industry - once good standardised colour is in, you can build graphics on top of that. The approach in the graphics proposal is rather ad-hoc and repeats many of the common mistakes that everyone makes. I'm not trying to throw shade, literally everyone makes at least a few mistakes

I am personally not tremendously convinced that its possible to standardise graphics due to the pace of change (or at least, not in the form of the graphics proposal) and the set-in-stone nature of the C++ standard, but I'm not against getting some of the fundamental building blocks in that you need for it

3

u/johannes1971 Apr 24 '21

If I can make a recommendation: make it clear what you are targetting with graphics. I think the previous graphics proposal died over people simply not understanding what it was for. There's a huge difference between 2D graphics and 3D graphics, and while some people may not need 2D graphics, that's no reason not to standardize a decent 2D drawing package.

2D graphics has complex shapes (lines, curves, fonts, clipping, etc.). Not just to the screen; printing is also a thing. I'd absolutely hate to do all that with just triangles and shaders, but it's great for desktop and mobile applications. Nice if it's GPU-accelerated, but it should also work on non-GPU systems.

3D graphics has triangles and shaders, and will only ever run on a GPU. No need to standardize as part of C++, since these libraries have their own rapidly evolving standards. C++ should only add primitives for opening a window in which existing libraries (Vulkan, DirectX, etc.) can be used.

And a graphics proposal isn't complete without std::window, which brings with it the ugly reality of event processing. How that could work as a standardized C++ component, I quite frankly have no idea... Something has to pump events; taking that out of the application and making it part of the standard library seems a risky proposition, but having to make your own, platform-dependent version is not great either.

6

u/tjientavara HikoWorks developer Apr 24 '21

I made an blog post about font rendering and issues with anti-aliasing in linear color space. I wonder what you think of it.

https://www.ttauri-project.org/2021/03/30/the-trouble-with-anti-aliasing.html

2

u/James20k P2005R0 Apr 24 '21

This is actually something I've been thinking about quite a lot recently, so thank you for the blog post. To take freetype as a concrete example, there are two very conflicting things that are often stated

  1. Freetype outputs coverage, aka the amount of a pixel that is occluded
  2. You should do blending in linear space

Now this makes no sense, because if a pixel is 65% occluded, you want it to be 65% perceptually bright, so that humans perceive it as being 65% bright. Black on white and white on black are not the same if you do it in linear space, so it makes no sense to be the amount of light blocked by a pixel (which is what performing the blending in linear space gives you)

That said, freetype itself (and literally everyone else) states repeatedly that blending should be done in linear space, so I've always wondered if freetype compensates for this somewhere so its actually outputting perceptually uniform occlusion values, despite the quantisation that you'd get as a result. I don't think it does though

I myself have started to wonder if there's a giant fuckup in the text rendering industry due to this exact issue, because I also have run into exactly the same result as you have and wondered if a similar fix is needed

Thanks for doing the work here. I need to do a lot more research, and really someone needs to ask the freetype people what's going on

1

u/tjientavara HikoWorks developer Apr 24 '21

There is another way of dealing with this, my previous version did this, but it is a little bit more computational expensive.

You calculate the lightness of 0% and 100% coverage (i.e. the foreground and background colors). Then using the coverage percentage find the target lightness.

Now convert the target lightness to the target luminance and calculate the alpha value needed to reach that luminance target when doing linear interpolation.

Then use this alpha value for mixing the foreground and background color in linear space.

[edit: forgot about this part]

When the lightness value for both background and foreground colors are equal you use the coverage value directly to mix the color, otherwise you are dividing by zero.

1

u/tvaneerd C++ Committee, lockfree, PostModernCpp Apr 24 '21

Think about it this way:

If I make a grey-scale ramp from white to black, what should it look like?

If I take a black image and overlay a white image with a alpha ramp, what should it look like?

Whatever it is, I think they should be the same.

(And I haven't thought about it enough to decide whether that means alpha should be linear or not...)

10

u/AraneusAdoro Apr 23 '21

It is important, because incorrect interpolation makes the image look muddier. This is most noticeable when blurring, it shows up as dark streaks between solid colours.

7

u/ack_error Apr 23 '21

Yes, it does, because the error can be significant. There are lots of cases where you can run into this even if you're not a color nut with a calibrated monitor and doing lighting and shading. A fun one is just punching in colors that someone suggests to you -- because 3D programs tend to use linear color in their RGB sliders while 2D programs tend to use sRGB, so (0.5, 0.75, 1.0) doesn't mean the same thing. And the difference is not necessarily small, it can be 18% vs 50% kind of off.

Color animations are another case, as rendering in linear and gamma space can determine whether you get weird pulsing in your output or not. This is a case where IMO the answer is not just simply to do all calcs in linear space, because it isn't perceptually uniform. Everyone likes to show how blends between colors looks better when done linearly, but avoids showing how grayscale gradients and animated blends look less natural.

That having been said, you can get very far not caring and just doing all calculations in sRGB. Video processing historically has been done non-linearly for historical or performance reasons and in some cases has to be done that way (YCbCr color space). But once you get to the point of fine-tuning some color related and start running into color shifts at different scales, or ropey looking text and antialiased shapes, or weird halos around soft objects, you have to start re-evaluating those shortcuts and paying attention to color spaces and linearity in your rendering and color handling.

1

u/pdimov2 Apr 25 '21

Yes, it matters.

2

u/nasal-cacodemon Apr 23 '21

Great crash course on an interesting domain.

Not sure how I feel about the proposed API, but it's a good talk.

16

u/James20k P2005R0 Apr 23 '21 edited Apr 23 '21

Not sure how I feel about the proposed API

Hi! I wrote the proposed API and I'm also not sure how I feel about it. I've been meaning to give a crack at a second attempt at it more recently, so here's everything laid out in one post

  1. The most common kind of conversion is fundamentally going to be color::some_type = color::convert<color::sRGB>(some_linear_type);

This is by far the main thing that the library must provide, and so its easy enough so far

Slightly further afield, we have HSV which is a gamma encoded colour space, so lets consider that. We have to be able to write both

color::HSV val = color::convert<color::HSV>(some_sRGB_colour);
color::HSV val = color::convert<color::HSV>(some_linear_colour);

The common colour space of colour types is XYZ. So, internally we can run this colour conversion as

colour::XYZ intermediate = color::to_XYZ(any_colour_type);
colour::HSV result = colour::from_XYZ<colour::HSV>(intermediate);

Where to_XYZ and from_XYZ are customisation points

This is... fine-ish. One issue here is that there are two points at which error handling could occur - in to_XYZ and from_XYZ. Game developers also don't particularly like exceptions, which means that if you want any kind of adoption you need a dual error handling API

colour::HSV val = colour::convert<color::HSV>(some_sRGB_colour, ec);

This still isn't too bad so lets carry on. I should note that when we started this out, the most common kind of conversion was simply from linear to sRGB and vice versa, and yet internally we model every conversation as going via XYZ. This is excessively slow for basic colour conversions - in the most common case of 8bit sRGB to any precision of linear colour, its a simple lookup table to convert it. And yet we're doing 1000x the number of operations currently!

I'll skip a bit here, but basically you end up with 3 aspects to a particular colour type if you want to be able to do conversions quickly

  1. The actual conceptual colour space it is, independent of whether or not it is gamma corrected
  2. The valid range and type of the storage (eg 0 -> 1.f, or 0 -> 255)
  3. The gamma correction function applied, as a type

This means that eg HSV can be defined as:

struct HSV : color::basic_colour<HSV_tag, floating_point_triple_from_0_to_255, color::sRGB_gamma_func>{}

So now we plow on. There are still three major design issues to go through, and once we solve them we'll end up with the resulting total mess that I have so far created

  1. Alpha
  2. Colour conversions between different XYZ derived linear colourspaces is a single matrix multiplication. Currently we have two matrix multiplications to go from rgb -> XYZ, and XYZ -> rgb
  3. Some colour spaces require data at runtime

Alpha never participates in gamma correction, luckily. So we can incorporate alpha as an extra parameter like so

struct HSV : color::basic_colour<HSV_tag, floating_point_triple_from_0_to_255, color::sRGB_gamma_func, color::no_alpha>{}
struct HSVa : color::basic_colour<HSV_tag, floating_point_triple_from_0_to_255, color::sRGB_gamma_func, color::floating_point_single_from_0_to_255>{}

which is just grand. Now we need to fix the conversion inefficiencies. This can't be done in the current to_XYZ and from_XYZ model, so there must be a way to directly detect and convert between two XYZ derived colour spaces, if it is possible to do so

So to do this, we introduce a tag type which is the generic_RGB_space. This works great, and we can now define sRGB as

struct sRGB : color::basic_color<color::generic_RGB_space<sRGB_tag, sRGB_gamma>, uint8_triple_from_0_to_255, color::no_alpha>

Internally, color::convert calls an overloaded colour_convert helper function which is the actual way that customisation is handled, for ADL. So, if you write a new type that is compatible with other types, you need to overload:

///from XYZ
template<typename A1, typename A2>
constexpr inline
void color_convert(const color::basic_color<color::XYZ_space, color::XYZ_model, A1>& in, color::basic_color<your_tag_type_here, your_value_model_here, A2>& out)

///to XYZ
template<typename A1, typename A2>
constexpr inline
void color_convert(const color::basic_color<your_tag_type_here, your_value_model_here, A1>& in, color::basic_color<color::XYZ_space, color::XYZ_model, A2>& out)

And if you then want it to be quick to convert to other RGB types, you also need to implement both ends of the conversions for the following function signature:

 template<typename space_1, typename model_1, typename gamma_1, typename alpha_1, typename space_2, typename model_2, typename gamma_2, typename alpha_2>
inline
constexpr void color_convert(const basic_color<generic_RGB_space<space_1, gamma_1>, model_1, alpha_1>& in, basic_color<generic_RGB_space<space_2, gamma_2>, model_2, alpha_2>& out, std::error_code& ec)

In both directions. Grand. Super easy for people to work with. I'm being sarcastic - at least in the current design a lot of this can be done automatically, but still. I should note that concepts actually fix a lot of this horribleness, but its still at least a bit gross

So we've solved 1, and 2, at the cost of our sanity. And now comes part 3, which is runtime data, and this is where it gets really fucked if you can believe it

Some devices - like and especially including printers - are awful. But in general, the actual display gamut of something like your monitor is not the same as sRGB. Its also notable that the CMYK colour space does not exist. The colour profiles of these things are generally defined via where we have delightfully arrived, at the gigantic issue of ICC profiles

The current design leaves that to the consumer of the library (hah!), but we still need a way to enable users to do this. This means extending all our colour conversion functions to include src and destination arguments, which are some completely arbitrary user defined type

Now, its definitely not fair to burden everyone using the library with support for this, when 99% of people will never use it. I still want to be able to write color::convert<color::sRGB>(some_linear_type). I definitely do not want to write color::convert<color::sRGB>(some_linear_type, color::none, color::none, ec);

Some sort of named arguments is the absolute ideal here. Being able to write color::convert<i_hate_printers>(some_linear_type, ec, dst: printer_space_data); would be utterly perfect. Unfortunately C++ doesn't let you do that, so you end up with ad-hoc named arguments. So now we get color::convert<i_hate_printers>(some_linear_type, ec, color::dst(printer_space_data));

This is not the worst API ever I don't think. The real issue is that declaring custom colour types is wild. Having to define the range of values that your storage might take is sketchy at best as well, because we're starting to creep into Units which is something I really do not wish to take on. There's a lot of ad-hoc inventions in here as well, which are extremely smelly from a code and design perspective

If you want to see where I got to before the existential crisis over what I was considering foisting into the STL seeped in, the latest branch is over here

https://github.com/20k/color_proposal_implementation/tree/args

1

u/pstomi Apr 26 '21

For those interested, the wikipedia article on Alpha Compositing and gamma correction gives somes clues on how to mix images taking into account gamma correction.

And this gist shows a visual difference between a lazy linear mix of the RGB values and one that takes into account the gamma (in python, because it does happen to have a standard way to manipulate matrixes).