Why is 3D Audio Needed in VR

For many VR applications, there may be awesome visuals in the scene, but it will be less immersive without accompanying audio. Therefore, the 3D audio experience is crucial to increase true immersion and sense of presence in VR environments.

There are a lot of factors related to audio experiences in VR applications, such as sound Localization, Room Reverberation, Distance, Spatialization, etc.

Localization

Localization is the inherent ability of human beings to figure out where a sound is coming from based on human auditory system. By understanding the human auditory system, real-world spatialized sound can be simulated in a VR environment.

Here, the factors are listed as:

  • Lateral:

    • Interaural time difference:

      The direction of a sound source can be determined by the time difference that a sound arrives to our ears, i.e. a sound wave is transmitted to the right ear before the left ear, and so the listener can be aware that the sound wave comes from the right-hand side.

    • Interaural level difference:

      There is also the sound level difference between both ears of the listener during sound propagation, i.e. the sound wave reaches the right ear before the left ear, the sound energy decays while sound is transmitting.

  • Elevation and Front/Back:

    In order to distinguish the direction of the sound which has the same arrival time and level, direction selective cues (pinna, head, shoulder, and torso) are used as the main reliable features. These cues are used to help determine where the sound comes from. Moreover, people can also cock their head up or turn their heads left or right to determine the direction of the sound source more precisely.

  • Pinna:

    When the sound arrives at the head, it will then be collected and filtered by the outer ears. These filters provide the feeling to the brain to determine where the sound is coming from.

  • Head/Shoulder/Torso:

    A sound wave is partially blocked or absorbed by the body of a listener during sound propagation. That means there is an inference of the body on sound feeling.

Room Reverberation

If the user is inside the room, the walls and objects in that room will reflect or scatter the sound waves. The sound waves will bounce again and again, and form a particular audio perceptual experience in that room. It not only builds a realistic feeling of the room, but also improves the accuracy of the directional perception.

Distance

The distance between a sound source and a listener is also an important factor on spatial audio application. There are some ways to describe the sound level decay that corresponds to the sound transmitting distance.

  • Loudness:

    Louder sounds are usually closer while quieter sounds are usually farther. In some cases, it is hard to judge the distance by the loudness if people are not familiar with the sound. The relative loudness is also considered. When a sound wave approaches, it sounds louder and vice versa.

  • Initial time delay:

    This is the time difference between the arrival of the direct sound and the first significant reflection, reflected from the surface of the room. When somebody is talking to a listener in a room, the listener will hear the direct voice right away then the reflected voice a bit later. If the person is standing farther away, the listener will probably hear the voice and reflections almost at the same time.

  • Mix of direct and indirect sounds:

    The human auditory system unconsciously mixes direct and indirect sounds. When the direct sound is perceived louder than the indirect sound, the sound is emitted near to the listener, otherwise the sound is far away from the listener.

  • Interaural level difference:

    When there is an obvious difference of loudness between the two ears of an audience, the sound is most likely to be close to the audience. When the loudness difference between the two ears of the audience is negligible, the sound is probably far away.

Spatialization

Spatialization is an audio effect that makes listeners feel immersed in a virtual 3D environment. Spatialization is a key factor for an immersive VR experience. A lot of features are accomplished to simulate the real spatial feeling in 3DSP Audio SDK, such as:

  • Head-related transfer functions (HRTF) recording and improvement

  • Higher Order Ambisonic simulation of sound direction

  • Room audio simulation

  • Adding background noise floor

  • Real-world acoustic properties of distance

  • Sound occlusion simulation

  • Hi-Res audio support

and many other features