3D Audio With Raycasting
Ойындар
The track is Punch Deck's latest outrun track - "Music To Wear Fingerless Gloves To" - • Punch Deck - Music To ...
-
There's a lot I've been experimenting with and working on lately so I'll try to summarise it as best as I can here.
When a sound plays, 484 rays are cast outwards from the sound source. Each ray bounces 8 times to determine the properties of the room around it, while also checking for line-of-sight with the listener.
Based on how many times a ray bounced before it achieved line-of-sight with the listener, I can determine how strong the echo and low-pass filters should be. For example a sound that plays around the corner should sound muffled to the player, but the reverb and echo should sound clear when it reaches the player.
When raycasting completes, low-pass filters and EAX reverb parameters are applied to the sound source with OpenAL-soft. Each frame afterwards, 32 rays are re-traced to check for changes in the environment or changes in listener position, and the OpenAL effect parameters are adjusted if necessary.
All raycasting logic is performed on a separate thread on the CPU, meaning this isn't something that's RTX specific.
I'm planning to write an article on this soon with diagrams and accompanying code; please let me know if you would be interested in reading it.
Пікірлер: 46
wow I've never heard sound that dynamic before in a game
@Vercidium
2 жыл бұрын
Thank you! Your compliment means a lot
@zennon_youtube
8 ай бұрын
I know this is two years ago, but I've stumbled across your channel and am really enjoying your stuff. Even multi-billion dollar companies don't even make stuff like this!@@Vercidium
@Vercidium
8 ай бұрын
@@zennon_youtube thank you for the kinds words, glad you're enjoying the videos! Yeah this one is pretty old, I have an up-to-date raytraced audio video planned but haven't started working on it yet. So many ideas for videos!
@Vercidium
6 ай бұрын
@@YourLocalCafe haha yep I'm still hanging around
OpenAL Soft + Raycasting?? I'm SOLD. Hope this can be implemented into more OpenAL games, or even the main OpenAL Soft branch someday. I'm very much looking forward to reading the article 👀
@Vercidium
2 жыл бұрын
Sweet! I plan on sharing it all soon, I've got some more techniques to try and things to optimise first.
You might be interested to know this is how the sound engineers for Nier Automata did dynamic reverb in their game. They used a similar ray casting technique and you can read about it on audiokinetic
@Vercidium
2 жыл бұрын
Thank you, will do!
That is quite incredible. The site recommended me some more recent video of yours and I was impressed by what you did. I don't really know what is going on, but I checked some more videos of yours and I really like this.
Nice, this new system is coming along well!
What a time to be alive
Holy cow! That opens up the door to SO many interesting puzzles!!
Great work!
Man that sounds so awesome!!
@Vercidium
2 жыл бұрын
Thanks Bouphen!
Sounds rly cool. I can tell that needs improvement, but sounds sick!!! Keep up with the good work
amazing! i wish there were audio plugins for music that can do that
Sick
Now the next neat thing to do is smell with raycasting! Wait...
@Vercidium
8 ай бұрын
Hahaha imagine that
It looks like YT removed my comment from yesterday (or I forgot to post it, who knows!), so I'm gonna reword it a bit: OMG, this is so cool. It reminds me of a geometric acoustics thingy that was implemented into Quake 3 except it ran on the GPU. Nice stuff. :3
@Vercidium
2 жыл бұрын
Thank you! I saw the notification for your comment then couldn’t find it so that makes sense haha
I'm 100% interested in reading about this and seeing code. How is the performance? A handful of sound sources wouldn't eat into performance too bad, but what about a hectic scene with explosions, fully automatic fire from dozens of NPCs/Players? How does it perform in outdoor settings? In forrest?
@Vercidium
2 жыл бұрын
It runs pretty well on my PC since all the heavy lifting is done on other threads, and more threads can be set up to distribute the workload if the user has the available cores. If there's more sounds playing than the user's PC can handle there will be a slight delay on sounds, but for the most part sounds initialise and play within 1ms. Playing a 12v12 match with bots works great for me but I know some players with older PCs might struggle, so I'll add sliders for ray amount + bounce amount, and the option to disable audio raycasting completely. Outdoor settings works well at the moment but I have some ideas for improving it.
Hey are you still going to publish that article on this stuff? I would love to read it, this is definitely one of the best sound systems I've ever seen
Quick question. Do you have any plans to make this sound system have a way to pass through thinner walls? A traditional sound system on top of it allowing the sound to pass through walls as well would add another layer of realism, but I’m not sure how it would work out in practice. This is already the best dynamic sound system I’ve ever seen. Can’t wait to see follow up videos
@Vercidium
5 ай бұрын
Thank you! Yes I implemented permeation where the strength of the muffling is affected by the thickness of the wall. I’m not sure I’ve shared a video of it though, I’ll have to post a new one
@axule4373
5 ай бұрын
That’s really cool. How did you go about calculating the permeation of the wall? And is it fully dynamic without any extra setup on the environment? I’d love to see a video explaining it
@Vercidium
5 ай бұрын
@@axule4373 the first time each ray bounces, another ray is cast directly towards each sound source, tallying up the total amount of blocks they pass through. The more blocks they passed through, the thicker the muffling This means you can also ‘press your ear’ against a wall by standing next to it, which makes it easier to hear through walls, because the first bounce of each ray would all be very close to your player
okay but that beat though
Is this actuate by your estimation or slightly exaggerated to make the concept apparent? Either way. Very dynamic. Super cool.
@Vercidium
2 жыл бұрын
I've increased the echo time and muffling so I can easily tell whether the system is working or not. My next step is to figure out how different materials (metal surface vs sand surface) should affect the final sound.
@Jhat
2 жыл бұрын
@@Vercidium really cool project!
@Vercidium
2 жыл бұрын
@@Jhat thank you!
woahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
Ah so you're inheriting HRTF from OpenALSoft, nice nice.
@Vercidium
8 ай бұрын
Yep OpenAL soft does all the sound processing magic. I was using an older version of it for a while, and it sounds incredible after upgrading to the latest
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOoooooooo!!!!!!!!!!!
Is speed of sound taken into account?
@Vercidium
4 ай бұрын
It is, OpenAL Soft takes care of that
@vibaj16
4 ай бұрын
@@Vercidium Very cool. How does your system compare to Steam Audio?
You could do a caramella dansen meme to promote the game on tik tok
@Vercidium
2 жыл бұрын
That's..... a really good idea
listen, just even model the ground plane. people don't listne, they don't understand. when a plane flies by, quickly duck your head towards the ground. you will hear an extreme change in the perceived "pitch" of the jet noise, because it is 100% produced by the ground plane delay. now, please tell every audio developer they are nothing compared to xoxos. facts.
Friday night funkin'