In VRTogether the users are holoported inside a three-dimensional scenario, and they can remotely participate in a social virtual experience.
The technology used to represent them is called Volumetric Video, and thanks to the most advanced real time compression and transmission systems, it is possible to transmit, over the internet, this novel video representation. However, given the early stage of its development, the quality of the volumetric video is not comparable to the one of the most known traditional 2D video and the VRTogether consortium is strongly focusing on improving every aspect of this technology.
For these reasons, i2CAT has created a solution called Point Cloud Multipoint Control Unit (PC-MCU), able to optimize the video content and minimize the effort of processing for the end-user machine. The PC-MCU acts as a cloud component: it receives the volumetric video from all the users involved in the experience, it processes them, applying smart quality selections techniques and, in the end, transmits to each user a personalized data flow, specifically adapted to the characteristics of their machines.
Thanks to the PC-MCU, the users will only receive the data incoming from their peers that they are observing at that moment, saving bandwidth and computational resources.
In addition, the PC-MCU is able to consider the relative distance between users in the virtual world. Depending on how far two users are placed, indeed, it may not be needed to receive a video at it’s full resolution. Thanks to a bidirectional transmission channel, the PC-MCU knows the position of the users in the three-dimensional scenario and, depending on the distance between such positions, can downgrade the quality and the resolution, considerably reducing the amount of data to be processed.
The following video shows the PC-MCU in action, together with a highlight of the optimization provided.
The initial results obtained by the PC-MCU has been successfully introduced in the ACM NOSSDAV’20 workshop within the ACM MMSys’20 Conference, presenting the paper PC-MCU: Point Cloud Multipoint Control Unit for Multi-user Holoconferencing Systems, winner of the Best Workshop Paper Award.
This project has been funded by the European Commission as part of the H2020 program, under the grant agreement 762111.