The ultimate goal of the Meta-Object research is to enable a transformative reality-virtuality convergence by facilitating seamless interactions that transcend spatial and temporal barriers. This is to be achieved by implementing next-generation virtual objects that inherit the form, properties, and functions of their real-world counterparts, allowing for smooth synchronization, interaction, and sharing between the physical and virtual worlds. This will all be realized within a post-metaverse intelligent simulation platform through wearable AR/MR/VR devices.

This research aims to develop an open XR collaboration platform that induces high-quality immersion and social co-presence to enhance user collaboration quality. The platform is based on Scene Graph and utilizes digital twins to manage collaborative interactions of spatial models, static/dynamic objects, sound, hand gestures, and haptic elements through functions like creation, deletion, movement, and transformation. The study focuses on developing high-quality immersive elements and enhancing social co-presence by emphasizing various technologies. The research covers the composition and integration of 3D objects in the spatial domain, optimal alignment of scanned data and CAD models, sound reproduction techniques, application of hand gestures using reinforcement learning in a physics simulator, high-quality multi-modal haptic technology supporting wearable interfaces, avatar mesh-point cloud interaction, force-based interaction, deep learning models for 3D clothing reconstruction, adaptive spatial configuration between multiple VR clients and single AR hosts, CG avatar face and costume reconstruction with texture extraction, as well as the development of evaluation metrics and factors for enhancing social co-presence in XR.

For the effective application of smart glasses to everyday life, anyone can use them intuitively, reflect surrounding situations and individual characteristics, and ultimately research and development of WISE (Wearable Interface for Sustainable Enhancement) beyond the functional use of the interface. Dynamic interfaces reflecting the overall state of the user’s attention and intention, all surrounding spaces and objects are recognized as semantic objects to access related information, and the object is used as an input tool for each characteristic.

In this project, we study a scene graph-based 3D scene reconstruction method for the dynamic digital twin. The dynamic digital twin track and update dynamic movement of physical objects in the indoor environment in real-time. In addition, the project studies XR interworking technology in which users in the field and remote users experience and manipulate the same environment directly or indirectly through realistic rendering-based augmented and virtual reality. Finally, this project will enable various realistic non-face-to-face collaboration applications. To implement this system, we intend to organically converge specialized technologies in various fields, such as computer vision, computer graphics, and indoor GIS.