At ACM CHI, CWI will give a talk (Thursday, May 09, 11:00 am) on the methodology used to define the subjective protocol for users’ QoE assessment in social VR application, and its application is a photo-sharing experience using social VR systems, as described in the paper “Measuring and Understanding Photo Sharing Experiences in Social Virtual Reality”. Find out more on our protocol by watching the video summarizing our contribution and reading our ACM CHI paper.

The results presented in these two high-quality venues address one of the objectives of the project: to develop appropriate Quality of Experience (QoE) metrics and evaluation methods to quantify the quality of these new social VR experiences. During the first year, the project has created a new protocol and set of metrics, and the associated data analysis toolset, for evaluating social VR with end-users. The impact of this result may go beyond the project, since it can be become the de-facto standardized manner for evaluating a new genre of experiences: social VR. The protocol and metrics include both quantitative and qualitative aspects, such as a new questionnaire combining presence, immersion, and togetherness; a set of objective metrics based on the behavior of the user, focusing on speech analysis, neck rotation, body movement, etc.; and performance metrics for profiling the system aspects. This novel evaluation method has been used for the evaluation of the pilot in October 2018, and has been iteratively developed and validated through a human-centered process, including two experiments (with around 100 users in total).