Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal data

Abstract

Abstract

This study examines the embodied ways in which learners monitor their cognition while learning about exponential functions in an immersive virtual reality (VR) based game, Pandemic by Prisms of Reality. Traditionally, metacognitive monitoring has been assessed through behavioural traces and verbalised instances. When learning in VR, learners are fully immersed in the learning environment, actively manipulating it based on affordances designed to support learning, offering insights into the relationship between physical interaction and metacognition. The study collected multimodal data from 15 participants, including think-aloud audio, bird's-eye view video recordings and physiological data. Metacognitive monitoring was analysed through qualitative coding of the think-aloud protocol, while movement was measured via optical flow analysis and cognitive load was assessed through heart rate variability analysis. The results revealed embodied metacognition by aligning the data to identify learners' physical states alongside their verbalised metacognition. The findings demonstrated a temporal interplay among cognitive load, metacognitive monitoring, and motion during VR-based learning. Specifically, cognitive load, indicated by the low- and high-frequency heart rate variability index, predicted instances of metacognitive monitoring, and monitoring predicted learners' motion while interacting with the VR environment. This study further provides future directions in understanding self-regulated learning processes during VR learning by utilizing multimodal data to inform real-time adaptive personalised support within these environments.