VRTogether project partner TNO participated at the 10th ACM Multimedia Systems Conference (MMSys 2019), held from June 18 – 21, 2019 in Amherst, MA, USA. MMSys is the leading international forum for researchers to present and share their latest research findings in multimedia systems. While research about specific aspects of multimedia systems are regularly published in the various proceedings and transactions of the networking, operating systems, real-time systems, databases, mobile computing, distributed systems, and middleware communities, MMSys aims to cut across these domains in the context of multimedia data types. This provides a unique opportunity to investigate the intersections and the interplay of the various approaches and solutions developed across these domains to deal with multimedia data types.

5G networks offer a substantial improvement over previous mobile network generations. As always for each new generation, more bandwidth is promised. But, 5G also offers improvement in other areas: ultra-low latency on the radio layer, distributed in-network processing with edge computing near the base stations, network slicing to offer improved quality guarantees, and a dynamic core network that integrates concepts such as SDN and NFV.

As such, 5G has much to offer for a variety of industries (also called ‘verticals’) and use cases. Virtual Reality and gaming are among the use case that are often mentioned. Both VR services and gaming require high performance and low latencies. For VR, low latencies are needed for immersion, as smooth and continuous playback needs to be guaranteed, also when using state-of-the-art streaming technologies that only stream the user’s current field of view. For gaming, the need for low latencies is clear to anyone who has played a real-time game.

For VR Communication services, such as being developed within the VR-Together consortium, these low latencies are also required end-to-end. Typically, the rule of thumb for end-to-end latencies required for good communication is 150 ms one-way, with up to 300 to 400 ms still being adequate for normal communication. For the VRTogether system for communication in VR, reaching such latencies is a real challenge, as we need to process all the captured images: for background removal, for HMD removal, for 3D reconstruction, for point cloud or time varying mesh encoding, and so on.

Thus, 5G networks may offer new infrastructure that can help building future communication services. Within 3GPP, many VR communication use cases are studied in the new 5G&XR study item (TR  26.928). The 1 ms latency on the radio layer will give a significant improvement already, but new technologies such as edge computing (e.g., for carrying out all this processing mentioned above) or slicing (for guaranteeing bandwidth and resource allocation during a communication session) may help achieve the requirements for these new immersive services.

The Dutch national news show for kids, Jeugdjournaal, recently explained new 5G networks based on the gaming use case. They brought 5 kids to the TNO Medialab to experience our shared VR experience as an example of how we build future immersive experiences. The enthusiasm of the kids was amazing, showing that the new generation is ready for a new generation of services and required infrastructure!