VR giant Oculus is to finance a year-long research project at TU/e to move closer than ever before to perfecting ‘virtual sound’. Lead researcher Maarten Hornikx explains the process, and describes his own ideas for spin-off applications of the technology, including a potential VR tool for the blind.
Most people familiar with VR systems will be aware of the leaps and bounds the visual side of the technology has made in recent years, in terms of building realistic virtual worlds, increasingly hard to tell apart from the real thing. Something we don’t always consider though is how much audio can contribute to the sense of being in a believable virtual environment.
For this reason, Oculus is funding research by Maarten Hornikx, assistant professor of Urban Acoustics at the TU/e, in order to move closer than ever before to perfecting virtual 3D acoustics.
“This research project is about investigating how close we can come to realism with virtual acoustics”, explains Hornikx. “You can grasp that by imagining that the sound you hear now – together with the effect of the room, because it is always influencing the way you perceive the sound – that it can be identically reproduced. So if you close your eyes and have headphones on it sounds the same as if you were in the real situation.”
Just as with 3D visuals, 3D audio is intended to totally immerse the participant in the action (whether gaming or watching a movie), making them feel like they’re really in the given situation. Traditionally this effect is achieved by placing a dummy head and torso within the recording environment, with two microphones embedded inside the dummy’s ‘ears’ picking up the sound. The benefit of this is that the recorded audio takes into account the physical influence that a human body (and indeed the shape of human ears) has on how the environmental sound is heard.
Listen below with headphones to experience a famous demo of 3D-sound, created by QSound Labs
The idea behind Hornikx and Oculus’ collaboration though, is to see if it is possible to model acoustics virtually. To do this, the research team will register the acoustic properties of every object and surface within a small office-style test environment, before using this information to calculate and build a computer model of the sound dynamics of the entire room.
At the same time, using a dummy, they will record the environment for real – before having test subjects listen for differences between the two. The goal is to see how close they can get the virtually assembled audio to sound like the real thing.
“I’m really excited by this project with Oculus”, says Hornikx. “They also have acousticians on board so for me it’s really easy to communicate with them, because we talk the same language. To my best understanding what they want is more understanding of the opportunities of this – how far we can go with virtual acoustics for use with those VR goggles”.
For Hornikx though, who has a background in researching how sound behaves in the urban environment, the knowledge gained from this research could open up various possibilities outside of improved gaming or movie experiences.
Amongst them is the possibility of developing a tool for architects and planners to experience the acoustics of buildings before they are built – a huge benefit when constructing not just concert halls and arenas, but residential or office buildings where the impact of sound is also an important consideration, but often an afterthought.
An additional use that Hornikx sees is for city planners to be able to generate virtual models of their ideas, and demonstrate them to the public.
“One of the most important procedures in handling noise is to communicate it with people who may be living somewhere where they are annoyed by noise or maybe have sleep disturbances”.
“If Eindhoven airport says they are thinking about more planes or another runway for example, people might think ‘Oh my God, now we will have even more noise’. But if you could make it audible to the people how it sounds in the future, then you have a way to communicate it”.
“Also, if there is a highway and the authorities plan to erect a sound barrier, the citizens might claim it’s not tall enough. But then an acoustician could make a virtual environment and say ‘yeah, but listen to the decibel reduction'”.
Perhaps even more interestingly, Hornikx has also been involved in the study of human ‘echolocalization’ – a method where blind people are able to perceive their surroundings through the reflection of clicking sounds made by themselves. He has hopes that by perfecting virtual audio, it will be possible to build a VR training tool, offering blind people the chance to train in this technique in a multitude of different virtual environments, and without facing the dangers of practicing it outdoors.
“There are benefits from such a virtual scenario. It’s really easy to change the situation: Maybe it’s too hard to detect the door opening, so you can simply change it to a wider virtual door”.
“Also if [blind people] test in the streets, they need to pay attention to sounds on the street. So this virtual training makes their life easier”.
Related Posts
