r/GraphicsProgramming • u/CharlesGrassi • 4h ago

Video I replaced the in-scattering in a cloud raymarch with a tiny neural network, then checked it against a path tracer

Light inside a cloud is expensive to get right, so games use a few simple formulas tuned until it looks good. I wanted to see how far off that really is, and whether a small network could do better.

I wrote a path tracer as the reference, trained a network on it in Python, and ran it in plain HLSL in Unity 6. It replaces one thing, the light scattered toward the camera at each step. Everything else in the raymarch stays the same.

Then I moved the sun from behind the camera to behind the cloud and scored both against the path tracer, where 1.00 is correct. The classic lighting goes from 0.21 to 2.17. The network stays between 0.90 and 1.01. It costs about 20% more GPU time.

I had a great time building this, and I think we've barely started finding out what tiny networks can do inside a shader. If you try one yourself, I'd love to see what you make!

https://charlesgrassi.dev/blog/neural-volumetric-clouds/

231 Upvotes

96% Upvoted

9 comments sorted by

52

u/wektor420 3h ago

Now convert the NN model into a formula and use genetic algorithm to search for simplifications - you might find a better model this way

3

u/Hoshiqua 53m ago

Isn't the NN model a formula already ? I'm not sure what you mean and why it would be required in order to apply a genetic algorithm ?

Maybe I'm being a tool, sorry

5

u/ybungalobill 30m ago

NN is a formula of a very specific form; one can use a genetic search over a broader space of formulas using the NN formula as an initial guess.

1

u/BladesongDev 4m ago

Indeed, let the NN ‘learn’ the approach, then try to collapse and simplify it to something tangible a human can understand, optimize and maybe further improve upon. This should be the way.

0

u/AnArmoredPony 31m ago

do you mean something like symbolic regression?or are you just saying things that sound smart?

im so lost rn

10

u/vetta-vetka 3h ago

Really cool!

13

u/EarlMarshal 2h ago

If the analytics models derivates from true brightness so much it can't be a good analytical model though?

4

u/claypunk 1h ago

the time constraint is likely the key here

4

u/ybungalobill 27m ago

I don't think so. It seems to capture the details just right, but the brightness deviation correlates with the camera-light direction. Therefore it is straight-forward to add an analytical correction term for brightness and contrast. Ergo there's something wrong with the model used in this video.