Fraunhofer IDMT and Fraunhofer IIS have developed the Fraunhofer Immersive Panner, which enables 3D audio productions for live applications with conventional production tools
Live production of audio content, whether for broadcast or streaming, requires not only specialized know-how, but also absolutely reliable equipment free from interference. But not every audio console in live production already boasts a 3D panner as required for 3D audio productions. To give production houses equipped with consoles for stereo or 5.1 surround productions the power to create 3D audio, the Fraunhofer Institute for Integrated Circuits IIS and the Fraunhofer Institute for Digital Media Technology IDMT have now developed the Fraunhofer Immersive Panner (FIP). Based on Fraunhofer IDMT’s proven SpatialSound Wave technology, which focuses on the production and playback of object-based sound installations for theater and entertainment, Fraunhofer IIS and Fraunhofer IDMT adapted the system for live broadcasts. This involved making changes specifically to the user interface and adding interfaces to common production infrastructure.
A flexible and modular solution
The FIP consists of a flexible hardware solution with versatile options for audio interfaces, be they MADI or audio over IP (AoIP). It allows three-dimensional panning of up to 64 audio objects and rendering to up to 32 output channels. The panned audio objects can be rendered either for immersive speaker systems or binaurally for stereo headphones. A wide range of speaker setups can be created in the 3D panner; the presets available for this can also be customized as desired. This means that in addition to standard 5.1, the FIP supports the creation of 3D audio formats such as 5.1+2H, 5.1+4H or 22.2 in live production. Advanced setting options such as adaptive bass management or individual limiters for output channels make it possible to create a 3D audio mix in real time. The browser-based, intuitive user interface is platform-agnostic. Its modular structure means it can be flexibly adapted to individual requirements and special production situations. The system can operate using any end device. Accessing the FIP via tablet, for example, brings up a user interface with multi-touch functionality. Because the FIP can be operated remotely via IP and web interface, several people can work on a shared immersive mix simultaneously from anywhere in the world. Especially given the trend toward remote productions during the coronavirus pandemic, the ability to mix from any location can help maintain distance between employees on-site and also keep the travel budget down.
Simple operation and optimum adjustment
In combination with an MPEG-H Audio authoring and monitoring unit, the 3D sound can be turned into a completely defined, object-based and interactive MPEG-H audio scene. On the user interface, the 3D space is defined from the perspective of the producer. Individual audio objects are created and positioned in this space. These individual objects or groups of them can be assigned sound properties as well as individual icons and colors. This makes it easy to keep track of individual actors or groups, even in complex productions. All settings can be created, saved and recalled in advance of a production. During a live broadcast, the audio objects can be moved around in the 3D space and positioned using an innovative panning concept. This makes it possible to, say, create a highly realistic listening experience by tracking the stage dynamics. For particularly user-friendly operation, certain parameters can be fixed, for example the distance of an audio object from the ground. This way, the FIP enables intuitive production of 3D audio content with existing infrastructure.
Header image: ©lisegagne – stock.adobe.com; Fraunhofer IIS