System for the Synchronisation of Position, Motion and Audio Between MR and VR

Date：March 2024

Teams:Kadinche

Writers:Soko Aoki,Author PictureNaoki Itabashi,Author PictureRipandy Adha

PDF:System for the Synchronisation of Position, Motion and Audio Between MR and VR

Abstract

The desire to share experiences and spaces with others is a behaviour based on human social and validation needs, and as digital technology develops, richer information tends to be shared Experiences in MR and VR are subjective experiences using head-mounted displays, but the technology to share these with others is still developing In particular, in MR the user is able to share information while looking at a real space. In particular, MR is an experience where additional digital information can be viewed while looking at a real space, but if this is to be communicated to others in remote locations, the space and the additional digital information need to be shared simultaneously. Therefore, in this research, a system has been constructed to allow remote VR users to experience the experience of MR users, with the aim of sharing the MR experience with a sense of realism. In this system, real-time synchronisation between MR and VR users is achieved, and the uploading and placing of 3D objects is realised, so that the experience of MR users is shared in real-time with VR users in remote locations. The system is designed so that the position, movement and sound of users in both MR and VR are synchronised and shared. The position, movement and voice of both MR and VR users are synchronised and shared. User tests have shown that MR-VR user collaboration makes users feel as if they are together without spatial separation, and that the webspace content editor is valuable when users want to make sense of and share the space. This paper presents the collaboration system and the web space content editor.