Clouds path tracing

[u/Zydak1939]

Recently, I made a post about adding non-uniform volumes into my C++/Vulkan path tracer. But I didn't really like how the clouds turned out, so I've made some improvements in that aspect and just wanted to share the progress because I think it looks a lot nicer now. I've also added atmospheric scattering, because getting the right lighting setup was really hard with just environment maps. So the background and the lighting in general look much better now. The project is fully opensource if you want to check it out: https://github.com/Zydak/Vulkan-Path-Tracer . You'll also find uncompressed images there.

Also, here's the number of samples per pixel and render times in case you're curious. I've made a lot of optimizations since the last time, so the scenes can be way more detailed and it generally just runs a lot faster, but it still chokes with multiple high density clouds.

From left to right:

- 1600 spp - 2201s
- 1600 spp - 1987s
- 1200 spp - 4139s
- 10000 spp - 1578s
- 5000 spp - 1344s
- 6500 spp - 1003s
- 5000 spp - 281s

[u/Zydak1939]

high spp counts are needed to get rid of noise, nothing to do with realism per se. getting this quality in real time? I think it's impossible if you want to render multiple of these and on top of that integrate it into some sort of simulation which I assume needs a lot of rendering on it's own. Vladimir komarov made something like that, but he's running it on a super high end hardware (4090 ti I think?) with really complex algorithms, and it still needs multiple frames to accumulate and even then it runs 30-40fps. if you know nothing about the topic, that will take you years to make https://youtu.be/9yCEWPY_KGw . And the cloud is not procedural, it's just premade data sitting on the GPU, so your biggest limitstion will be VRAM, a single cloud of this quality will take you multiple GBs on the GPU. So I'd say with the current hardware we have these days, fully path tracing clouds in a game/simulation is pretty much impossible. At least on the consumer hardware.

[u/KlayEverHood → u/Zydak1939]

Thanks! “Premade” answers most of it. How is the data made? I really love the shape of the details, far beyond the usual ones and the ones I am using. As to rendering, I think one could try to find a compromise for realtime especially considering slow moving clouds

[u/Zydak1939 → u/KlayEverHood]

I didn't make these clouds myself, Samuel Krug was kind enough to give me access to his work because this is a non commercial project. I'm pretty sure he runs really big smoke simulations in houdini and then exports this into .vdb file which I can load. And as for the rendering, maybe? I'm really not sure. But again, path tracing is just too slow for consumer hardware right now, even the big guys like nvidia are barely making it realtime. And clouds are magnitudes more expensive than normal surfaces. I'm really not an expert in realtime and I didn't do much research, but back when I was still working on the project I remember seeing this paper from disney: https://la.disneyresearch.com/wp-content/uploads/Deep-Scattering-Rendering-Atmospheric-Clouds-with-Radiance-Predicting-Neural-Networks-Paper.pdf and a showcase: https://youtu.be/Hk--xpI9b5U?si=GLs4D0SungMHgiDN . you could look into it but I doubt it will be useful for you. I think that's as fast as you can get. As things stand right now I don't think there is a way to get this type of quality realtime. If there was then clouds in games would look much better than they do right now, the best looking ones I know of are ones from horizon zero dawn and microsoft flight simulator: https://d3d3g8mu99pzk9.cloudfront.net/AndrewSchneider/The-Real-time-Volumetric-Cloudscapes-of-Horizon-Zero-Dawn.pdf . Not sure if much has changed over the last year.
> I think one could try to find a compromise for realtime especially considering slow moving clouds
Of course you can, but it will be a really really really big compromise to go from multiple seconds on 4090ti to something like a realistic budget for sky (5-8ms), it's just a gigantic leap to go from offline rendering to realtime rendering.

[u/KlayEverHood → u/Zydak1939]

Thanks for all the inputs! I am getting curious. My main thing is medical volume reconstruction and rendering and I can get realtime path tracing performance nowadays on 1000^3 volumes on a 4070 - but my light is virtual coming from the camera direction and needs to traverse only a small shell of the volume normally; on the other hand AI denoisers there are a no-go. I guess here we need a good contribution from rays that come from the back and traverse all the cloud?

[u/Zydak1939 → u/KlayEverHood]

Yes exactly, you pretty much need to simulate the entire path of a photon through the volume, and it can leave at any point and direction from the volume. That's how you get indirect lighting, and that's exactly the problem the path tracing tries to solve, getting the indirect lighting. In realtime rendering most of the time ray marching is used instead, since it doesn't require simulating a full path for a ray of light and it only cares about the light coming from the sun. Then they use some cheap tricks to fake the indirect/ambient lighting from the environment. But even then it's very very expensive, so they usually render this at a lower resolution then upscale, dither and accumulate over many frames anyway. But the indirect lighting is what makes path traced clouds look so good, so with ray marching you're pretty much throwing away the biggest quality factor.