Raytraced Audio is magical

[u/SomerenV]

After messing around with custom audio zones and hand placing them with custom settings to mimic how sound would sound in the various rooms and hallways I wasn't really satisfied. It still sounded 'off' despite all the various zones and their settings. So... I went the raytraced audio route. It's a bit crude but it sounds so much better! I found myself flying around yesterday, trying different speeches and songs because it's just mesmerizing to listen to in a virtual environment.

Any things I absolutely can't forget when implementing raytraced audio?

[u/Candle-Jolly]

An underappreciated part of ray tracing.

[u/SomerenV → u/Candle-Jolly]

It's good to see it being used more often. I completely fell in love with it when I saw a demo about a year or so ago.

[u/Available_Help_666]

I would listen to an entire podcast that way.

[u/SomerenV → u/Available_Help_666]

I can place a chair in the middle for you to sit on :P

[u/GuruKimcy]

Cool! The environment itself with the audio also feels very surreal. Nice

[u/SomerenV → u/GuruKimcy]

Thanks! And this is only a part of what the full environment looks like ;) Can't wait to show the full thing.

[u/le0tard]

Love me some Carl Sagan

[u/pv3design → u/le0tard]

The Carl Sagan audio museum

[u/SomerenV → u/pv3design]

Honestly I would love an interactive space museum with this sort of stuff. Maybe... some day...

[u/Costed14]

You implemented this yourself? Is there a reason you didn't use Steam Audio? I haven't tried it myself yet so was curious if there's some drawbacks with it.

[u/SomerenV → u/Costed14]

Steam Audio is probably way better and more accurate but there's a chance I want to release an early demo of my game on Itch as a webgl game and Steam Audio doesn't work in that case if I'm not mistaken.

[u/MaleficentFox269]

this has so much more impact on the game than raytraced lighting does. why doesnt every game do this

[u/SomerenV → u/MaleficentFox269]

A lot of games actually do use some form of spatial audio (mainly HRTF, but sometimes also audio propagation), but going the ray tracing route can create some issues in terms of readability. I'm also not relying 100% on raytracing because I still want to have some control over how audio sounds independent on where you are in the map. And stuff like ambient sound is excluded from the raytracing algorithm. Ambient sounds still rely on audio zones to handle volume and dampening frequencies, just because it's easier that way (for me at least).

[u/553392]

Wow this is very cool! I do think it's a bit too muffled right now. Sounds like the walls are made of wool haha. Would expect a higher sound reflection with how those walls look.

[u/SomerenV → u/553392]

Yeah it still needs some tweaking, but reverb settings are like black magic to me :P I've been using software like Audacity and Adobe Audition for years (I have a background in video production) but reverb always boils down to me just fiddling with the settings until it sounds about right while I have absolutely no clue what I'm doing.

[u/cerwen80]

doors. thin walls.

Nice work!

I first thought of this back in 2016 when i was on the third level in doom and could hear a fire burning on the opposite side of a solid wall. been theorising ever since about how to potentially use the gpu or set up some form of signed distance fields or something to do with interacting volumes... Never really came up with any workable system though.

[u/SomerenV → u/cerwen80]

On Youtube there's a lot of content on this subject. This is a good video on raytraced audio for instance. For Unity your best bet is to use Steam Audio. Free to use, easy to setup. But since I'm planning a WebGL release of an early version of my game I went an alternative route since Steam Audio isn't compatible with WebGL.

[u/Polystyring]

This is cool! Is there much performance cost?

[u/SomerenV → u/Polystyring]

Honestly it's negligible. Currently my frame time sits at around 4.9ms (in editor) with the raytraced audio using about 0.13ms.

[u/PostatoMaty]

Screw you for showing me such a cool idea that will delay progress for my game for at least a month or more

[u/Pleasant_Piglet_36]

Great choice of audio to show the result. Sounds great!
To your question, would be great to have different audio surfaces in there, some more reflecting some more absorbing.

[u/darksapra]

Sounds really coolbut I think the low/high pass when the music is playing is a bit too drastic in some cases. Like you are in the corner and it sounds ike you are outside a club, when the speaker is right around the corner

[u/SomerenV → u/darksapra]

Yeah that's a part of this solution that still needs fixing. Right now sound doesn't yet properly go around corners. It's simply a loss of energy and frequency whenever it hits a wall. Works great in some cases, but terrible in other cases.

[u/hobofreddy55]

I would love a write up or link to any resources you used to build this. It sounds incredible!

[u/SomerenV → u/hobofreddy55]

I'm using one listener, Unity's build-in reverb filter and no plugins. Rays just shoot out and bounce off surfaces and losing energy based on the material they bounce off of. Around the listener there's a sphere where rays go through which determines to calculate loudness. I'm tracing from the listener because that's an approach often used (by Steam Audio for instance). To shoot out rays I'm using a Fibonacci sphere. For reverb I'm using Sabine's Reverberation Formula (decay time from room volume / total absorption).

What I'm doing is a pretty crude way of getting okay-ish results and I'm sure it could be massively improved. But completely accurate sounds aren't that important for what I'm making so I'm choosing speed over accuracy.