Toward a comprehensive XR validation framework: insights from University of Glasgow: Alan Matthews
Since the explosion of consumer extended reality (XR) in 2016 the University of Glasgow has invested into XR provisions for both teaching and research. Our XR Teaching Lab is situated outside of the university at Partick Burgh Hall. This dedicated and permanent XR teaching space was created as part of Project Mobius (2018-2021), with the intention of exploring how immersive technology can be harnessed to develop new approaches to teaching to the benefit of the learning experience of students. This included the development of 10 exemplar lessons representing the integration of XR leaning into a breadth of courses across the university. We continue to include new VR learning interventions as the technology develops.
Over 1200 students every year make use of the XR Teaching Lab and some of the most commonly used apps are those exemplars developed as part of project Mobius. The continued success of the XR Teaching Lab is an example of the establishment of XR as an integral part of modern higher education. Over the past decade, researchers and educators have shown increasing interest in developing and evaluating XR tools for education. As the technology has been developing at pace, the modalities of its use in education have changed in turn, and so have the ways in which these tools are understood, deployed, and tested. To date, there is no single prevailing standardised approach to deploying or determining the efficacy of XR systems for educational ends.
Variations in the accepted definitions of XR, the wealth of pedagogical approaches, and the reliance on qualitative feedback makes it difficult to establish a baseline understanding of what works in XR teaching interventions and what does not. Specific XR tools and interventions need to be assessed on their own merits following a reproducible methodology grounded in appropriate ontological assumptions. We began by carrying out a review to examine:
• How are XR tools defined and their affordances understood?
• How is success or effectiveness, or usefulness measured?
• What pedagogical approaches or theories of learning are used as a guide?
• What real-life factors impact the outcomes?
We argue that there should be a reproducible comprehensive approach to running validation studies on XR tools.
Our project, Toward a Comprehensive XR Validation Framework, is currently underway with the aim of developing a robust and easy to implement validation framework for assessing the efficacy of XR teaching interventions, and then to apply this framework to three XR interventions, in the first instance the Project Mobius apps. The XR-TIE project consists of two strands. The first strand, currently in progress, will establish a novel methodology for assessing the pedagogical efficacy of XR interventions. This framework draws on existing research on the validation of teaching interventions and adapts it for the specialised context of immersive technology. The second strand of the project is designed to apply the XR-TIE framework to three specific XR interventions at the University of Glasgow.
