VRTogether aims at being experienced by users with different devices and infrastructure.
The consortium has then developed two distinct platforms to provide efficient solutions in a variety of scenarios, depending on the available resources and objectives.
The Web VRTogether platform is lightweight, does not require installations at the client side, makes use of simple capturing systems, and makes use of fully web based technologies.
The Native VRTogether platform requires the installation of a client application, and is able to support different end-user’s representation formats, including volumetric ones (TVMs, and Point Clouds), more advanced interaction features, more advanced capturing systems, media experiences seamlessly integrating heterogeneous media content, and higher scalability.
The figures below provide the hardware and software architectures of these platforms.
In detail we can separate the Web Platform into 3 individual web server instances, namely:
- TogetherVR web service: The TogetherVR web service is the main entry point for a client. It offers a web page to be accessible for a client to download all necessary resources and logic to create, display and facilitate the VR experience to the user. This also includes the connection to the orchestrator and Signalmaster.
- TogetherVR web orchestrator: The orchestrator is mainly a message forwarding entity that allows for the client to get all session related information and metadata necessary to execute and maintain a running VR communication experience.
- WebRTC Signalmaster: The Signalmaster is a standard WebRTC component (open source and not developed in the context of VRTogether) but necessary to initiate the WebRTC connections between clients (including NAT traversal and video protocol agreement).
The native pipeline integrates all the components required to build the collaborative platform which, in the end, allows users to participate in the VRTogether experience. End-users are captured in real-time and are represented in realistic and even volumetric formats, in a fully operational virtual environment. The native pipeline scenario allows to consume video in different formats (traditional 2D, stereoscopic, 360) which are delivered On-Demand or Live, which for this last one offers the possibility for the users to interact with the actors.
The Native Platform supports a variety of end-users’ representation formats:
- TVM: Real-time 3D Time Varying Meshes (TVMs) integrated inside a graphics environment like Unity.
- PC: Volumetric Point Clouds (PCs) via geometry-based (G-PCC) encoding.
- 3D Avatar
- 2D representations from the webcam.
- No visual representation (with or without audio interaction capabilities)