Interacting with holographic content in AR

Abstract

Different interaction methods are used in augmented reality to control the virtual content. Hand tracking, eye tracking, and audio commands are the three major interaction methods. In this thesis, we aim to assess the learning curve of hand interaction usage in handling holographic content.

We designed a Unity application for the Meta Quest Pro headset. We implemented two tasks for the users to complete. Additionally we made a tutorial phase before the actual experiment. The tasks involved grabbing, rotating, translating, and transforming holographic objects.

Our hypothesis for the project was that the task completion times between two study experiment sessions would show observable improvement during gameplay. In addition, to measure the learning curve, we collected and analyzed the total number of interactions, as well as the amount of missed and successful interactions.

The data collection was conducted in a following experimental session, held at the University of Oulu. The experiment was run with ten participants. Basic demographic information and possible limitations of using the video-passthrough feature were examined in the pre-questionnaire paper. During the playtesting, we focused on observing the improvement in the participants’ performance in terms of task completion times and total successful hand interactions.

The primary results and insights were examined through the questionnaire responses and data collection from the user experiments. Additionally, test users provided verbal feedback which helped us gain a new perspective on the results.

Looking at the results, the median task completion times in the second gameplay session were decreased almost in half when compared to the first session. This suggests that users can quickly learn the hand interaction mechanics, which supports the original research objectives and hypothesis. To verify this assumption, it is recommended, in the future studies, to conduct multiple gameplay sessions for each participant instead of just two sessions. This would verify the intentionality and accuracy of hand interactions, thereby revealing the potential utility of this approach for various applications in the future.