With the march of 3D film and television, immersive virtual reality and gaming, it's about time for a real step forward in the quality of spatial audio. And we think we've achieved that.
Our TOA professional studio tools and Rapture3D high-end playback and gaming products have a common way of doing what they do. If you're interested in how they work, read on...
Once upon a time, there was mono. Spatially, this was nice and simple - sounds just came from your speaker.
Later on, the "stereo" that we know so well came along. Largely developed by Alan Blumlein, this encodes a simple sound stage into two channels of audio, typically by varying the relative level of sounds in two speakers.
More recently, 5.1 and 7.1 "surround sound" systems have appeared, in the cinema and then the home. These put speakers around the audience and work typically in much the same way as stereo, varying the level of sounds in the two closest speakers.
So, What Now?
Firstly, height. We live in a 3D world - sounds come from above and below as well as the front, back and sides, both directly and indirectly. 5.1 and 7.1 systems don't manage this, which isn't that realistic. Okay, they beat stereo, but we can do better.
And also, there's been a lot of research done into how sounds behave in space. It turns out that just varying the level of sound in two speakers is not the best way to produce a realistic soundfield.
The Mathematical "Heavy Lifting"
Rapture3D and the TOA plugins uses a mathematical framework known as "Higher Order Ambisonics" (HOA) to drive your speakers. The maths is hard, but we worry about all that - and the results are worth it! HOA can work better and better as you add more detail and more speakers and, like its close relative "Wavefield Synthesis", can generate essentially correct soundfields over an area. Unlike Wavefield however, it doesn't insist on huge numbers of speakers. Instead, Rapture3D does the best it can and blends in more robust techniques where there isn't enough detail, or a large enough number of speakers, for more "holographic" approaches.
We think we have a superb HOA playback system ("decoder") in Rapture3D. We're using a range of new techniques, including handling of irregular speaker layouts, management of wavefront curvature - and new psychoacoustic cues. If you're mathematically inclined, you might be interested in some Technical Notes.
Rapture3D also has support for surround stereo, HCTC crosstalk cancelled stereo, and 3D over stereo headphones, using six different "HRTFs" to help you find one that matches your head shape.
Of course, the Rapture3D game engine does more than just play back HOA: it mixes the HOA content in 3D itself, automatically and live!
In the studio, the TOA plugins provide a rich suite of creative tools to help you make mixes using HOA easily. TOA stands for "Third Order Ambisonics" and needs 16 audio to channels to work. This gives a good level of spatial detail without overloading your computer.
What Does This All Mean?
This means that Rapture3D can:
- Make better use of your existing multichannel speaker layout to produce a more realistic and immersive soundfield.
- Handle 3D speaker layouts and place sounds above and below, as well as to the front, back and sides.
- Use "irregular" speaker layouts - tell the software where your speakers are and Rapture3D will work out the best way to use them ("advanced" edition only).
- Produce "binaural" 3D on headphones.
- Drive stereo and surround stereo systems.
Rapture3D can be used for virtual reality, computer gaming and music playback (where mixes have been done this way). The Rapture3D "User" edition works with your standard Windows speaker layout. The Rapture3D Advanced Edition unleashes the full flexibility of the customisable Rapture3D decoder and includes VST plugins which make these playback techniques available in professional studio software.
The TOA plugins are entirely aimed at the studio. These have a wide range of creative possibilities. For instance, with these plugins you can:
- Place any number of sounds in any direction in 3D space.
- "Upmix" content from existing formats (e.g. stereo or 5.1) into 2D or 3D mixes.
- Work with real 3D recordings from ambisonic microphones.
- See the soundfield graphically so there is no confusion about what's going on.
- Manipulate 3D mixes using rotation, reflection, movement, reverb, spatial EQ and much more.
- Use a single TOA master to produce many output mixes such as stereo, binaural, 5.1, 7.1, Auro-3D, 22.2 and more.
You know you want to...