Physics-based Acoustics

Summary

In this episode of the Immersive Audio Podcast, Oliver Kadel, Monica Bolles and Bjørn Jacobsen are joined by a Principal Dev Leader at the Mixed Reality division at Microsft Noel Cross and Senior Principal Researcher at Microsoft Research Nikunj Raghuvanshi from Redmond, US.

Nikunj likes to invent techniques that create immersive sight and sound from computation. He is endlessly fascinated with simulating the laws of physics in real-time and finds it thrilling to search for simple algorithms that unfold into complex physical behaviour. He has over a decade of research and development experience at the intersection of computational audio, graphics, and physics, with over fifty papers and patents. His inventions have been successfully deployed in the industry, particularly Project Acoustics, which is bringing immersive sound propagation to many major AAA game franchises today. Nikunj is currently a Senior Principal Researcher at Microsoft Research. Previously, he initiated interactive sound simulation research at UNC-Chapel Hill during his PhD studies, whose codebase was acquired by Microsoft. 

Noel grew up playing games on my Commodore 64 and Amiga computers. His love for multimedia computing helped him to start working at Microsoft as an intern in 1991 in the multimedia team. This was the age of the SoundBlaster 16 ISA cards and CD-ROMs were just being introduced into PCs.  Out of the multimedia team, the DirectX team was born to accelerate the development of high-quality games for the PC. Noel worked on DirectSound and audio drivers for Windows getting a taste of the game development community attending several GDCs in the 90s. This was the first time he was introduced to 3D audio algorithms and at the time the technology didn’t impress much.  Through the 2000s, he worked on every release of Windows with the focus on improving the audio subsystem. This led to the complete overhaul of the audio infrastructure on the Windows Vista platform which has remained largely intact since introduced in 2006.  The most current stop on his Microsoft journey is working on Mixed Reality devices. He worked on the speech and audio functionality exposed from HoloLens and Windows Mixed Reality devices with a concentration on spatial audio. After having lackluster impact in the 90s with spatial audio, he’s been reinvigorated working on this technology with the introduction of high-quality HRTFs and head-tracking services to complete the experience. Spatial audio processing has also led Noel to better understand the impact of acoustics on virtual 3d worlds. His team is currently working on Project Acoustics which allows developers of 3d titles to take advantage of wave-based simulations to handle how audio propagates in the real world.

In this episode, Nikunj and Noel dive deep into the topic of physics-based virtual acoustics along with Project Triton and Project Acoustics covering fundamental theory, research, technology and case studies.

Listen to Podcast

Show Notes

Project Acoustics: Making Waves with Triton – https://youtu.be/pIzwo-MxCC8

Gears of War 4, Project Triton: Pre-Computed Environmental Wave Acoustics  – https://youtu.be/qCUEGvIgco8

NOTAM – https://notam.no/meetups/the-ambisonics-salon/ 

Nikunj Raghuvanshi’s LinkedIn: https://www.linkedin.com/in/nikunj-raghuvanshi-a499172b/

Noel Cross’s LinkedIn: https://www.linkedin.com/in/noel-cross-0b9a51167/

Microsoft Soundscape – https://www.microsoft.com/en-us/research/product/soundscape/

Microsoft HoloLens – https://www.microsoft.com/en-us/hololens

AVAR 2022: AES 4th International Conference on Audio for Virtual and Augmented Reality – https://aes2.org/contributions/avar-2022/

Senua’s Saga: Hellblade II – Gameplay reveal – https://youtu.be/fukYzbthEVU //

https://twitter.com/dagadi/status/1470000580223504389)

Directional Sources & Listeners in Interactive Sound Propagation using Reciprocal Wave Field Coding – https://www.youtube.com/watch?v=pvWlCQGZpz4

​​Project Acoustics | Game Developers Conference 2019- https://youtu.be/uY4G-GUAQIE

Interactive sound simulation: Rendering immersive soundscapes in games and virtual reality – https://youtu.be/2sKPDGBsM0Q

Project Triton Research website – https://www.microsoft.com/en-us/research/project/project-triton/

Notes on Parametric Wave Field Coding for Precomputed Sound Propagation – https://www.microsoft.com/en-us/research/publication/parametric-wave-field-coding-precomputed-sound-propagation/

Survey

We want to hear from you! We really value our community and would appreciate it if you would take our very quick survey and help us make the Immersive Audio Podcast even better: surveymonkey.co.uk/r/3Y9B2MJ Thank you!

Credits

This episode was produced by Oliver Kadel and Emma Rees and included music by Rhythm Scott.