Comparative performance analysis of an XR digital twin prototype under AR/VR and movement conditions using various performance metrics

Abstract

Extended reality (XR) technologies have been improving rapidly in recent years and accessibility is beginning to be at a very reasonable level with the release of the Meta Quest 3. As the hardware becomes more attainable for consumers and for companies in large amounts, now is a good time to explore different software solutions for these technologies. This thesis investigates the development and evaluation of a digital twin application focused on meeting rooms that allows users to inspect and control a physical space through augmented reality (AR) and virtual reality (VR) using interactive sensor data visualization features. By doing this, it enables the reader to better understand the development, limitations, and evaluation of single platform XR applications involving internet of things (IoT) data.

Before starting the planning of the prototype, related work was explored. This included investigating and testing state-of-the art XR hardware, testing games and applications that featured both AR and VR aspects, as well as examining research papers about similar prototypes. This proved to be an effective way to better understand requirements and limitations of these technologies.

After the implementation of the prototype, a technical evaluation was performed to find and understand performance differences of the prototype under different conditions, using frames per second (FPS), frame time, total render time, stale frames, as well as random access memory (RAM), graphics processing unit (GPU) and central processing unit (CPU) usages. The examined conditions included AR, VR, and VR with a passthrough camera view (VR-PT) modes, all with and without added head movement during the measurements.

From analyzing the collected data, we found that AR performed significantly better than VR and VR-PT modes. VR and VR-PT modes performed quite similarly, with VR-PT performing slightly worse due to the added strain from the passthrough camera. We also came to the conclusion that developers of similar XR applications should include frame time and total render time data in the evaluation of the application, because short-lasting changes in performance are not as visible in FPS data.