Adoption of virtual reality technology in higher education: An evaluation of five teaching semesters in a purpose-designed laboratory.

Virtual and augmented (VAR) technology is in the early stages of being adopted as a teaching platform in higher education. The technology can facilitate immersive learning in environments that are not usually physically accessible to students via 3D models and interactive 360° videos. To date, adoption rates of VAR technology for teaching have not been well described across a higher education institution. Further, there is an absence of information on the optimal VAR laboratory designs and cost per student. In this study, a purpose designed virtual reality laboratory was formed in 2017 at The University of Sydney, housing 26 Oculus Rift headset units. An evaluation was conducted on the design, costs, rates of teaching adoption and student experiences over five teaching periods (2.5 years). Over this period, 4833 students were taught in the laboratory across 7952 student visits. The laboratory was used most by the Faculty of Engineering (53%), followed by the Faculty of Arts & Social Science (23.8%) and Faculty of Science (23.2%). For engineering, the units of study using the laboratory represented only 1.4% of annual faculty subjects offered. This confirms that adoption was in the initial stage of innovation diffusion. The laboratory saw a 250% increase in student numbers over the period of evaluation and 71.5% of students surveyed (n = 295) reported enhanced learning outcomes. The cost per visit was only AU$ 19.50. These findings give confidence to higher education institutions that the right VAR technology infrastructure is a sound educational investment for the future. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10639-021-10653-6.

Introduction

Virtual and augmented reality (VAR) technology is one of the most ground-breaking technologies of this century, and higher education is in the early stages of adopting the innovative technology as a teaching tool (Achuthan et al., 2020; Grivokostopoulou et al., 2020). The up-take has been led by science, technology, engineering and mathematics (STEM) disciplines, on the back of rapid industry demand for workforces of the future (Hernandez-de-Menendez et al., 2020; Kaminska et al., 2019). There are numerous ways in which VAR technology can be employed to deliver immersive learning experiences: augmented reality projections, virtual constructed 3D realities and interactive 360° videos. The technology has the ability to take students to locations that are physically restricted (construction sites or urban informal settlements). It allows interaction with 3D models or designs in orientations and scales not previously possible. Competencies generated for students from VAR in education include: spatial visualisation, innovative thinking, problem-solving, critical thinking and empathy (Hernandez-de-Menendez et al., 2020; Kaminska et al., 2017). The exciting possibilities for education are only restricted by the imagination of the creator.

Adoption of technology innovations in education has been described using a number of frameworks, including Roger’s innovation diffusion theory (Straub, 2009). For virtual laboratories important factors for adoption have been identified as; technology acceptance, relative advantage, intention of use and trialability (Achuthan et al., 2020). Trialability is positively correlated with the rates of innovation adoption as it allows the user to experience and experiment with the technology firsthand (Rogers, 1995). It follows that the opportunity to have hands-on experience with VAR technology is central to an educator’s willingness to adopt the technology in their teaching. This is especially important as there are a large range of teaching pedagogies that can be used with VAR technology (Grivokostopoulou et al., 2020). VAR technology on its own does not ensure positive learning outcomes (Reeves & Crippen, 2020). A review of 25 virtual laboratory studies in engineering and science reported that 13 studies concluded that there were no effects or negative effects on students learning outcomes (Reeves & Crippen, 2020). Poor quality teaching content is a particular problem, characterised by inadequate realistic dynamic interactions (Potkonjak et al., 2016) and limited multi-sensory experiences (Portman et al., 2015). For educators in higher education, is it critical that they develop appropriate teaching content by learning from hands-on experience and the positive examples of VAR education in their disciplines, for example in engineering; civil engineering (Shirazi & Behzadan, 2015), environmental engineering (Grivokostopoulou et al., 2020), mechanical and electrical engineering (Kaminska et al., 2017).

As adoption rates of VAR technology in higher education are linked to an educator’s ability to experiment with VAR teaching content, it is important that the right technology infrastructure is built. However, universities are reluctant to invest in new infrastructure if they do not have a pre-confirmed number of users that will ensure value for money. There is presently a gap in the research about rates of adoption of VAR technology across a higher education institution, as VAR infrastructure is generally designed for a single educational purpose or discipline (Kaminska et al., 2019). VAR technology can cause discomfort, mainly in the form of motion sickness and headaches when using headsets (Cassani et al., 2020), hence it is critical that the right technology is selected for the learning cases. Further, poor educational experiences can be exacerbated by inadequate employment of the technology; such as limiting the number of head-sets per class due to cost (Kaminska et al., 2019) and visual lag due to poor internet connectivity. There is a lack of published research on the optimal designs and costs for VAR teaching technologies and an absence of data from longitudinal studies of innovation adoption across entire higher education institutions. This absence of information makes it difficult for educational institutions to make informed decisions about VAR technology and to justify large centralised learning investments.

In 2017, The University of Sydney invested in a purpose built virtual reality (VR) laboratory that was accessible to the whole university. At the time it was built, it was the largest VR lab in Australia. The aim of this research was to conduct a detailed evaluation of i) the technology choices, costs and physical space considerations of the laboratory, and ii) the rates of teaching adoption by faculty and student experiences across five full teaching semesters (2.5 years).

Methods

Virtual reality laboratory design

The virtual reality (VR) laboratory was designed by the authors for their own innovative teaching needs. The space was designed for tutorial teaching and fitted with the latest VR technology. Tutorials are small group teaching and generally range in size from 15 to 30 students. In early 2017, a critique of the technology platforms available identified both the Oculus Rift and the HTC Vive as industry leading. To meet the tutorial class size requirements, space per device needed to be optimised. The HTC Vive was a headset with an attached cable that enabled a standing experience, requiring an estimated floor space of 4 m2. The Oculus Rift was also cabled but offered a good seated experience with an efficient floor space use (2 m2). Oculus had a more diverse range of available content and had been recently purchased by Facebook. For these reasons, Oculus Rift was selected. There were additional criteria behind the VR technology selection that have been presented in-detail elsewhere (Ijaz et al., 2017). A computer laboratory was refitted to support 26 Oculus Rift devices (Fig. 1). Two HTC Vive units were also configured in pods for development use. Stable rotating chairs were obtained, that had no side arms or rollers. Disposable single user face-shields for inserting into the headsets were provided to users. The computers had the following software as a standard configuration: Oculus, Unity, and Steam VR. All costs for both the set-up and operation were collated and used to calculate the cost per visit in Australian dollars (AU$). The laboratory was named The Immersive Learning Laboratory (Thomas et al., 2019). At the time of its construction it was the largest laboratory of its type in Australia (Ijaz et al., 2017).

Fig. 1

The Immersive Learning Laboratory with a virtual reality tutorial in progress. All students are exploring content using Oculus Rift headsets

Adoption of virtual reality teaching

Teaching in the laboratory commenced in semester 2 (S2), 2017. The laboratory was based in the Faculty of Engineering but was accessible, free of charge, for teaching classes and trialing content to all staff across the University. Academics and professional staff who wanted to use the laboratory underwent a brief training course. Further, any internal and external visitors who wanted a trial session in the laboratory were accommodated around teaching sessions. Bookings were managed by the lab team, which included a part-time laboratory technician. Academics and their tutors were responsible for the conduct of their unit’s tutorial or other teaching activities. The laboratory technician was available if required for technical support. The laboratory team conducted on-going operational and maintenance tasks to keep the laboratory running. The numbers of classes and students was captured in the booking system by subject and by faculty. To measure innovation adoption the portion of units using the lab was calculated for a given year for the Faculty of Engineering. Teaching in the laboratory was suspended in semester 1 (S1) 2020, due to the Covid-19 pandemic.

Content creation

To support academics to create original teaching content, the right skills, software and hardware were needed. Lab technicians were employed with skills in 360° video filming and 3D content creation (generally in Unity, Blender and ThreeJS). Hardware purchased for 360° filming included a GoPro 360° Max camera, Kandao Obsidian stereo 360° camera and SP360 Drone mounted with a Kodak PixPro 4 K 360° camera. Additional software was provided for content editing: Adobe Creative Suite. The laboratory encouraged student research projects to develop unique content.

Student experience evaluation

An anonymous short online survey was used to evaluate students’ experiences in the laboratory. Broadly the questions covered: students’ prior experience with VR, reflection on their learning experiences, if they felt any discomfort and if they wanted to use VR in the future in other units of study. The full survey is included in the supplementary material (Online Resource 1). Participation in the survey was voluntary and individual academics decided if they wanted to allow time for their classes to participate. Informed consent was obtained from all individual participants in this study. Engineering student responses were analysed in detail, as they represented the largest cohort of users.

Data analysis

Booking system data capturing usage rates (both teaching and visitors) was collated annually. Any variations or anomalies were cross-checked by the lab team with the academic who made the booking. For example, if all bookings were used as booked and if the student numbers were accurate. Student survey data was downloaded and cleaned annually. Quantitative data was visualized and had descriptive statistics run in GraphPad Prism. Qualitative data was analysed and coded thematically in Nvivo software (QSR International).

Results

Virtual reality laboratory design and implementation

The initial investment to implement the VR laboratory in 2017 was AU$ 117 540 (Table 1). The greatest cost portion (66%) was the 26 high-powered computers (AU$ 78 000). There was a large quantity of in-kind institutional costs, in the form of time from academic staff and Information & Communication Technology (ICT) technical team that has not been monetarised. Further, for this particular set-up there were minimal costs with respect to refurbishment of the existing space. Additional costs of purchasing 360° cameras have not been included. The annual operations costs are also explained in detail in the subsequent section (Table 2).

Table 1

Virtual reality laboratory hardware and signage implementation costs in 2017

Budget items	Details	Cost	Total
		AU$	AU$
VR units	Oculus Rift × 26	$ 850 ea	$ 22 100
	HTC Vive × 2	$ 1350 ea	$ 2 700
Computers	Custom towers × 26	$ 3000 ea	$ 78 000
Ancillary items	Monitors, key-boards, mouses × 26	$ 200 set	$ 5 200
Stable chairs	Refurbished × 24	$ 285 ea	$ 6 840
Signage	External and internal	$ 2700	$ 2 700
Total			$ 117 540

Table 2

Annual operational budget for the VR laboratory cover two teaching semesters

Operational budget	Details	Cost	Total
		AU$	AU$
Technician (casual)	20 h × 26 weeks	$ 51.50/hr	$ 26 780
Replacement batteries	AA rechargeable × 80	$ 6.50 ea	$ 520
Face-shields (disposable)	3000 student visits	$ 0.50 ea	$ 1500
Repairs to headsets	Ear-piece pair × 5	$ 150 ea	$ 750
Total			$ 29 550

The laboratory layout was workable with 24 VR units, however most classes used only 20 units or less. The space was previously designed as an ICT service centre with secure swipe access, a lower ceiling and no windows that could open. The re-fit for the VR lab saw increased number of computers and people in the space and this resulted in significantly more heat being generated. Unbeknown during the refit, was that the air-conditioning unit in the laboratory did not have a sensor and the main air-conditioning system sensors for the building’s floor set the temperature. The consequences of this were that the laboratory became very hot and stuffy, especially in the winter months when the main air-conditioning system was on heating mode. An assessment by an air-conditioning technician concluded that the only solution was the installation of a new system at significant capital cost.

A consequence of re-fitting a space that was not previously used for teaching was the lack of ICT facilities common to university teaching spaces. The laboratory had no audiovisual systems (microphones, projector or screen). This greatly reduced the range of classic pedagogical instructional modes common in tutorials and led to some classes becoming disjointed with students not following instructions or falling behind. Work arounds were found which included sharing detailed instructions or presentation material on each student’s desktop computer for them to follow. To assist with audio issues, it was more effective to give instructions and feedback to small groups (rows of three VR units) rather than try and address the whole class. At the time of lab fit-out networking Oculus Rifts through a shared license was not a possibility, and computers had to be managed individually. This remains a draw-back of the teaching platform.

Software and content sharing experiences

Sharing of VR content for each tutorial was also hampered by not having networked computers. If new software was needed for new content, then it had to be individually downloaded to every computer prior to the tutorial. Additionally, if content had to be downloaded or streamed from elsewhere, then the internet connectivity was not fast enough for all users to be streaming or downloading simultaneously. This led to frequent crashes of the content during tutorials. As a work-around a common shared network drive (samba device) was made available for content to be accessed by students and this could be loaded by the staff on a single unit before the class. This meant that the content was already downloaded and available. This solution was not suitable for all tutorials as some of the interactive content was only available on applications that were streamed, such as Google Earth VR.

Laboratory operation and maintenance requirements

Operation of the laboratory required a part-time staff member working approximately 20 h per week during the teaching weeks. The main tasks included: replacing consumables (hand-controller AA rechargeable batteries, face-shields), organising bookings and training, clearing file storage and desktops, uploading new software and content and recalibrating the Oculus sensors. The laboratory has one of the highest reported densities of Oculus Rift headsets for a teaching space. While the sensors were orientated to meet the minimum space requirements, it was found that there were frequent frame-shifts. That is the two sensors tracking the hand-controllers and head-set lost their spatial orientation. This would result in the frame-of-view for the student shifting randomly in any plane, such as a complete 180° reversal so the student had to face with their back to the sensors in order to see the forward view of the content. This would occur for at least two out of the 24 units after a day of teaching in the laboratory. To fix the problem the unit needed to have the sensor area manually recalibrated. Possible causes of these frame-shift were investigated but no clear reason was identified and hence staff continued to reactively respond to this issue.

Once every week, the laboratory was cleaned by the technical staff and all headsets and controllers wiped down with anti-microbial wipes. Once a month, swipe access requests for the lab were processed and bookings were placed for more detailed cleaning (carpet vacuuming and dusting). After two semesters of continual use the most common points of failure with the Oculus Rift head-sets were ear pieces being broken off and the audio failing to work (even if ear pieces were not broken). The headsets were refurbished in early 2020 with new foam head-piece inserts and replacement ear-pieces purchased from Oculus. At the end of 2020 the headsets are due to the be replaced after 3.5 years of use. The computers had a lifespan of four years and are due to be replaced by mid 2021.

Operational budget was needed for technician’s wages, replacement batteries (rechargeable), disposable face-shields and replacement head-set parts. This total cost was generally AU$ 29 550 for two teaching semesters (one calendar year)(Table 2).

Adoption of virtual reality laboratory teaching

Up-take of teaching in the laboratory by academics was strong. In the first semester the laboratory was opened (S2, 2017) 658 individual students completed a total of 1764 visits (Fig. 2). Seven units of study were taught, with four of them from engineering. There was greater demand in semester 2 (S2) than S1, due to demand by larger units of study. The most heavily utilised teaching semester was S2 of 2019, when 1674 individual students (1825 student visits) across 14 units of study were taught in the laboratory. Compared to S2, 2017, this was a 250% increase in individual students taught and evidence of a real increase in adoption rates. The total student visits only increased slightly. This was due, in-part, to the laboratory becoming increasingly booked-out during popular teaching times. Overall, the laboratory reached a large number of students over the five teaching semesters, with 4833 individual students taught through 7952 student visits. Also, the laboratory recorded 1860 visitors (both internal and external) during the period of evaulation.

Fig. 2

Numbers of students taught in the virtual reality laboratory from semester 2 2017 until semester 2 2019. Individual students were recorded (n = 4833) and their total laboratory visits (n = 7952)

Over the five teaching semesters, engineering units of study used the laboratory more than other faculties (Fig. 3). Of all the student visits, 53.0% (4212 students, 20 units) were from engineering units, 23.8% (1894 students, 16 units) arts & social sciences units and 23.2% (1846 students, 15 units) from science units. The greater uptake in engineering was likely linked to the physical location of the laboratory in the engineering precinct. This allowed for multiple tutorials in the laboratory to be planned for a semester. To look at relative innovation adoption, the engineering units in 2018 were assessed: nine individual engineering units of study with 13 unit codes (as some were co-badged for both undergraduate and postgraduate). This represented only 1.4% of the ~ 900 courses run annually in the Faculty of Engineering.

Fig. 3

The total numbers of students taught in the virtual reality laboratory by university faculty. Individual students were recorded (n = 4833) and the total laboratory visits (n = 7952)

Content delivered

Quality teaching is dependent on quality teaching content. Presented is a snap-shot of content type for seven engineering units of study, which used the laboratory regularly (Table 3). The majority of units (four out of seven) designed and developed their own content for teaching. That content included filming 360° videos using the laboratory equipment and developing new 3D models using the skills of the laboratory staff and available software. The other subjects either accessed free-online content, paid for subscription content or had content shared by industry. Independent of the content, the VR tutorial allowed students to be immersed in diverse and mostly inaccessible environments. For CIVL3310: Humanitarian Engineering, the laboratory was used to immerse students in 360° videos of select case-studies of communities in poor rural areas and informal urban settlements, in some of the least developed parts of the word (Thomas et al., 2017). The teaching pedagogy used collaborative learning based on the case-studies to develop competencies in critical thinking and empathy. In the case of ENGG5103: Safety Systems & Risk Analysis, the VR tutorial allowed them to safely conduct individual learning of lab hazards through interactive 360° videos. This developed student competencies in problem-solving. For CIVL3206: Steel Structures 1, students were able to individually manipulate 3D models of structures, where working with physical models of any real scale can present safety risks. This teaching case developed the student’s spatial visualisation skills along with innovative thinking. Details of content delivered by other units from other faculties is contained in the laboratory annual report (Thomas et al., 2019).

Table 3

Seven engineering units of study taught in the virtual reality laboratory. Immersive content used for tutorials and the origin of this content

Course code	Content used	Origin
AERO3760: Space Engineering 2	Student created satellite control system simulation in Unreal	Subscription content
CIVL2010: Environmental Engineering	360° videos of environmental engineering projects, interactive models of air pollution	Created by staff
CIVL3206: Steel Structures 1	3D models of steel structures, 360° of structures lab experiments	Created by lab staff
CIVL3310: Humanitarian Engineering	360° videos of community case-studies in developing countries	Free online
CIVL5277: Structural Rehab & Timber Design	Interactive 3D models of bridges for structural assessment	Created by lab staff
ELEC5518: Internet of Things for Critical Infrastructure	The application of virtual reality for design, construction & collaboration	Industry content
ENGG5103: Safety Systems & Risk Analysis	Custom 360° videos recorded of laboratory and work space risks	Created by staff

Student experience

Over the five teaching semesters, 295 undergraduate (UG) engineering students completed the online survey. The majority of the students were in the 3rd (41.9%) or 4th year (31.2%) of their undergraduate degree programs. For most students, it was their first VR experience (211 students, 71.5%, Table 4). Reflecting on if they felt the VR tutorial enhanced their learning outcomes for the unit 211 students (71.5%) responded positively, and 84 students said it did not. Over one third of the students (107, 36.3%) reported discomfort using the technology. The most reported discomforts were: headaches, dizziness, blurred vision, the weight of the head-set and that it did not fit over glasses. When asked if students wanted to use the technology in future units of study, 202 students responded positively (68.5%). From the positively coded student reflections, three key themes repeated; the excitement of using a new technology, enhanced content understanding and increased engagement in the tutorial. A selection of positively coded reflection quotes from surveyed students:

Table 4

Survey results from a selection of engineering undergraduate students (n = 295) who were taught in the laboratory

Survey questions	Yes		No
	n	%	n	%
Have you used VR technology previously?	84	28.5%	211	71.5%
Did it improve your learning outcomes for your unit?	211	71.5%	84	28.5%
Did you experience any discomfort using the technology?	107	36.3%	188	63.7%
Would you like to use this technology in other units in the future?	202	68.5%	93	31.5%

“Really cool and I feel fortunate using these hi-tech teaching materials for learning.”

(theme: new technology excitement)

“I feel that VR is going to advance so rapidly in the future and will be able to really enhance our learning. I don’t think a lot of people would have used VR and this will be a way to really get students to engage with their subject matter.”

(theme: new technology excitement)

“The program was extremely useful, it provided an easy and intuitive way to explore a structure. It felt real and applicable, and the ability to see the structure from all angles and at a very high level of detail was much better than seeing something similar on paper.”

(theme: enhanced understanding)

“It created a kind of immersive experience which blocked off outer distractions and forced you to put more attention to the content. Also the combination of audio and video learning helps create deeper impressions.”

(theme: increased engagement)

There was a reasonable lack of acceptance of the technology with 93 students (31.5%) responding that they did not want to use the technology in future units of study. From the negatively coded reflections, the following three key themes were identified: discomfort in using the technology, a sense that the quality of content did not enhance their learning and that it was difficult to navigate. A selection of negatively coded quotations from students:

“The quality in some of the 360 degree videos wasn’t great and caused some motion sickness feelings.”

(theme: discomfort)

“Turning around all the time was annoying and the goggles was a bit uncomfortable.”

(theme: discomfort)

“I believe that some of the 360 degree videos would have had the same result if being shown in class on a normal screen.”

(theme: no enhanced learning)

“The text on the frame was blurry and the models would occasionally break which deterred from the experience.”

(theme: difficult to navigate)

“Not being able to wear glasses during the lab. I heard the other classes could have their glasses on but our tutor said to take them off in the first lab, so I followed anyway. I couldn’t see very clearly and was always nervous if I clicked on the wrong app/video. Also had to squint my eyes to see clearer and always ended up feeling dizzy and uncomfortable after class.”

(themes: discomfort and difficult to navigate)

The survey data was further analysed to see if discomfort had an impact on if students reported a positive learning experience or willingness to use it in the future. Exploring if discomfort impacted negatively on reported learning outcomes, a similar ratio was found in both learning outcomes (yes and no) (Table 5). Calculated odds ratio (OR) gave near equivalence (OR = 0.96, 95% CI = 0.57–1.63, p = 0.89), which indicates that discomfort did not have a significant effect on learning outcomes. For discomfort and willingness to use the technology in the future, there was a higher proportion of discomfort reported for the positive future use (yes) compared to negative (no). This was evident from a calculated odds ratio of 1.73 (95% CI = 1.01—2.88, p = 0.05) where students who reported discomfort were significantly more likely to report positive willingness to use the technology in the future. Hence, discomfort was not a factor in learning outcomes or acceptance of the technology.

Table 5

Relationships between reported VR discomfort and learning related questions for engineering undergraduate students (n = 295)

		Improved learning outcomes?		Odds ratio	Future use of VR in units?		Odds ratio
		Yes	No	(p)a	Yes	No	(p)a
Did you experience VR discomfort?	Yes	76 (25.8%)	31 (10.5%)	0.96 (0.89)	81 (27.5%)	26 (8.8%)	1.73 (0.05)
	No	135 (45.8%)	53 (18.0%)		121 (41.0%)	67 (22.7%)

ap-values from the Fisher exact test for the contingency table

Cost per visit analysis

VR laboratories for teaching are an investment for any institution. The universal question is whether they offer value for money. To assist in answering this question, a cost per student visit was calculated. Over 2.5 years of operation the estimated cash expenditure was AU$ 191 145 (set-up AU$ 117 540 + (operations 2.5 yr x AU$ 29 550)). Divided by the number of student educational visits (n = 7952) gives ~ AU$ 24 per student visit. The laboratory was also used for numerous demonstrations and training activities for industry engagement and outreach activities (such as high-school student visits). In total, 1860 recorded visitors passed through the lab during 2.5 years of operation. If these numbers are included then the cost per visit to the laboratory becomes only AU$ 19.50. If the laboratory was operational in 2020, then there would have been a further reduction in the per visit cost, as the VR hardware was still within its working lifetime (3 – 4 years) and the operational costs were stable. Overall, the laboratory presented a good investment for the university due to the high number of visits it received.

Discussion

Laboratory implementation and operation

The implementation of the laboratory was successful as it was operational within eight months from (S2, 2017) the grant money being awarded. Common points for delay in the implementation of VR laboratories include having to wait for availability of VR hardware after new releases. Hence, for teaching laboratories it is recommended to use hardware that are already on the market and proven. The operational costs for the laboratory were kept as low as possible. A laboratory technician, with short-notice availability, was critical to the successful functioning of the laboratory. There were periods when a technician was not employed and the laboratory failed to operate effectively. That is despite all the academic users being trained and the availability of detailed instructions. The main points of failure were hand-controller batteries not being recharged and academics or tutors not recalling how to fix VR software or hardware issues. Like many teaching spaces in universities, on-call ICT support is needed to support smooth delivery of teaching content.

Another aspect of the laboratory design that needs to be revisited is the supporting teaching ICT equipment (microphones, projectors and screens). The absence of this equipment was a draw-back for teaching purposes and many academics were frustrated by the lack of these common teaching tools. The solutions are to install these elements in the laboratory or use VR technology based solutions for class co-ordination and information dissemination. One solution would be to build in better teaching navigation tools into the content, either embedded or in a supplementary form. The use of ‘embodied pedagogical agents’, such as embodied avatars, enhance the personal and social interaction and have been shown to improve student learning experiences, engagement and knowledge performance (Grivokostopoulou et al., 2020).

The cost of the laboratory was kept as low-as-feasible. This resulted in a very reasonable cost per laboratory visit (AU$ 19.50). There is little published literature on the exact costs of VR laboratories, despite there being reviews of different types of VR laboratories reporting that cost efficiencies can be a barrier to adoption (Potkonjak et al., 2016). It is anticipated that the cost figures presented in this study will be a useful planning tool for other educational institutions looking to invest in the technology. The availability of per visit cost data will allow ease of comparisons with other types of teaching spaces, especially those that VR labs might be able to supplement, such as wet laboratories for experimental work.

Adoption of teaching in the laboratory

Over the 2.5 years of reported operation, there was an impressive number of student visits (n = 7952) and individual visitors (n = 1860). Engineering subjects used the laboratory the most but only represented 1.4% of the total subjects in the faculty in a given year. Based on Roger’s innovation diffusion theory this represents the very first stage of innovation adoption, where the first to uptake are the ‘innovators’ representing 2.5% of any given group (Rogers, 1995). It has been reported that STEM subjects generally lead technology uptake (Hernandez-de-Menendez et al., 2020; Kaminska et al., 2019). However, quantification of this at the university level was not attempted in this study, and the usage rates by the Faculty of Arts & Social Science were strong, suggesting broader uptake by innovative educators across the university.

Although the total visit numbers were a positive indicator of adoption, it was originally projected that more individual subjects would use the laboratory. In total, there were 51 individual subjects taught in the laboratory, however, many of these were repeat subjects across the different teaching semesters. Total unique subject codes were only 24 subjects. While the repeat use by a subject give weight to the educational benefits of using the laboratory, the limited number of different subjects is evidence of barriers to use. There were numerous inquiries from academics who wanted to use the laboratory but did not have access to the right teaching content. Typically, the teaching content they wanted, or had envisioned, had not been created and they did not have the capacity (time, skills and/or finances) to solicit the creation of that content. To overcome this problem, in-house VR content creation skills were needed. During operation in 2019, laboratory technicians were recruited based on their ability to create VR content. This allowed them to develop bespoke content for individual academics, however, the time taken to fully develop a suite of VR teaching activities was greater than anticipated. Part of the lessons learned in developing content was the need for some best practice pedagogical guidelines for tutorial design for the laboratory. Other researchers who have developed teaching content for civil engineering undergraduate students in China, also emphasise the need to use a pedagogical framework that allows for experiential and discovery learning, reflective observation, abstract conceptualization and active experimentation (Walker et al., 2020). It has been recommended that academics and VAR researchers need to work closely to develop design guidelines and proven pedagogical methodologies (Ijaz et al., 2018).

This study has demonstrated that providing VR laboratory facilities across the university provided the first stage of adopters, innovative educators, a place to trial and implement their VR teaching. It follows from innovation diffusion theory that the next stage in adoption will come from ‘early adopters’ (13.5% of a given group) (Rogers, 1995). In the 2.5 years of this study we saw clear evidence of increased adoption, with a 250% increase in individual student visits in S2, 2019 compared to S2, 2017. Another indication of innovation uptake was the high number of visitors (n = 1860) who trialed the laboratory, validating the importance of trialability. It was anticipated that this diffusion of innovation would continue, had it not been for Covid-19 restrictions requiring us to close the laboratory in 2020.

Student learning experience

For the majority (71.5%, Table 4) of the engineering UG students surveyed it was their first VR experience, however that is likely to change as VAR technology becomes more mainstream for home entertainment. It was surprising overall, that there was a high portion (31.5%, Table 4) of surveyed engineering students who reported not wanting to use the technology in the future. These students were communicating their resistance to adopt the technology. This had not been anticipated and it points to some key issues with the student learning experiences that need to be further understood. The immediate notion suggested was that those students who experience discomfort might be reporting more negatively, however, this was not the case. Actually, it was the reverse, students who experienced discomfort reported wanting to use the technology significantly more (OR 1.73, p = 0.05). One explanation for this is that if students were not able to complete the VR learning experience, say due to discomfort of wearing glasses, they may want to attempt to use it again with the hope that the technology might have improved and they will have a more positive experience and keep-up with the class. The VAR industry is working to reduce the experience of motion sickness by incorporating additional brain-machine interface sensors in head-mounted devices, that provide more realistic immersive experiences and can respond if motion sickness is detected (Cassani et al., 2020). Haptic feedback sensors for the body is another area of development that will assist in reducing motion sickness (Potkonjak et al., 2016). These technology developments will assist in reducing the discomfort barriers for student but acceptance of VR in education by all students is likely to take time.

One explanation for the high proportion (31.5%) of students who did not want to use the technology in the future could be due to the teaching content quality. The student survey responses (n = 295) came from eight different UG engineering subjects, and due to the survey design the data could not be disaggregate by which subject the students had completed. It is highly plausible that students who reported not wanting to use the technology in the future might have experienced content that was not optimally designed. For example, some of the early civil construction 3D models have very limited interactivity and flaws with user navigation which allowed for rapid ‘uncontrolled flying’ out of the frame of reference. Additional observational research in the laboratory confirmed that navigating content and instructions was a challenge; students became ‘lost’ in the instructions and their ability to keep up influenced their learning rate (Hadigheh et al., 2019). Further evaluation is needed of both the student’s learning experience and the impact on learning outcomes for individual subjects. Researchers have emphasised the need for evaluation feedback loops to be integrated into any teaching using VAR, in order to facilitate improvement (Portman et al., 2015). Evaluations need to be designed that assess competencies of both the cognitive outcomes (knowledge and understanding) and the non-cognitive outcomes (motivation, social interactions and cognitive loads), with reference to student’s individual background and learning styles (Reeves & Crippen, 2020).

Future of VR teaching

The impacts of Covid-19 has required a rapid adaptation to distance learning and new ways to use VAR technology (Estrada & Prasolova-Forland, 2021). Transitioning the technology for individual student learning is currently being explored. VAR technology has the ability to be successfully deployed to every individual learner, to enable them to engage realistically in laboratory learning experiences, which are especially important in science and engineering pedagogies (Dietrich et al., 2020; Potkonjak et al., 2016). The different approaches being considered include supplying each student with a basic VR head-set and creating more VR content, especially those that replicate laboratory based learning. With good design, it is anticipated that VR can partially breach the divide between face-to-face teaching and the current distance learning deficiencies.

Conclusion

Higher education institutions have an important position in our societies to aid in innovation diffusion. This study has demonstrated there is the necessary adoption rates of innovative VR teaching at a university wide level, to fully utilize a purpose built VR laboratory. Further, that the provision of the VR technology contributes to the technology diffusion process and is a cost-effective investment for a higher education institution. To enhance innovation up-take it is important to provide in-house content creation support for educators. VR training and educational frameworks should be developed and delivered to ensure that quality content is delivered. Positive experiences of VR technology will allow our students to carry on the innovation diffusion process as they move to the workplace.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 132 KB)