Cultural and social participation through a joint virtual experience of real stage events
Public events are a vital part of cultural life. Interestingly, they provide significantly more sensory sensations than just watching and listening to what happens on the stage. An important part of the individual experience is the presence of others in the audience with whom the evoked emotions can be shared. Unfortunately, there are several factors preventing people from attending public events – either personal ones, like physical distance, or global ones, like a pandemic. This calls for a contemporary technical solution that goes beyond traditional formats.
While various efforts are being made to transport stage performances to safe and comfortable listening and viewing environments by the means of immersive technology (see our article on VirtualLiVe), SocialSTAGE-VR focuses on the aspect of cultural participation as a virtual social experience. Funded by the German Federal Ministry of Education and Research, the project is coordinated by Technische Universität Chemnitz with Fraunhofer IIS, vr-on GmbH, Die Etagen GmbH, point omega Aktiengesellschaft, and YOUSE GmbH as its partners.
On the technical side, delivering a lifelike experience into peoples’ living rooms independent of time and place relies on providing content for head-mounted VR displays and headphones. This way, every audience member can enjoy immersive sound and vision in stellar quality and from an ideal angle – no more bad seats or people in front of your blocking your view of the stage. Besides putting everybody in the front row, SocialSTAGE-VR also does away with audience number limitations – even the tiniest club concert can be made available to a virtually unlimited number of viewers.
So much for the action on stage, but what about the social interaction? SocialSTAGE-VR sets itself apart from other concepts by offering multi-user capability. Avatars representing the audience members interact with one another – verbally, as in a traditional chat, as well as non-verbally through gestures or even facial expressions. This transforms the viewer from being an anonymous witness to becoming part of a crowd and participating in lifelike social interactions in a way that feels completely natural. Apart from making the experience more realistic, the technology is expected to inspire artistic production and spark new approaches to audience participation.
In the audio field, Fraunhofer IIS utilizes its experience as an audio technology innovator by contributing a library for spatial sound rendering. With sound coming from the expected directions, the listener’s ears can fully immerse in the illusion of actually being at the event. As for the visual part, Fraunhofer IIS experts are working on practical ways to record live content as easily as possible so that it can be viewed with VR goggles while permitting the viewer to move in a certain radius around their virtual seat. This is important for the interactive aspect, enabling the viewer not only to look at their co-audience, but also to virtually lean over toward others. Due to the live nature of the recorded events, all audiovisual algorithms are required to – and will – work in real time.
By addressing the subject of social interaction, SocialSTAGE-VR is more than just another way of sharing immersive recordings of live events. It opens up new ways of experiencing interactive cultural events together with family and friends or even total strangers – regardless of questions of personal health, schedules or distance.
Header image: © Nürnberger Symphoniker/ Fraunhofer IIS
This post is also available in: Deutsch