Virtually perfect? VR as a tool for safety training
Creating a virtual training environment is now popular in multiple sectors, from construction to catering. But how do the outcomes stack up against real-world and screen-based learning?
First published in Health and Safety at Work Magazine, April 2019. Images added 2021.
Barriers and possible benefits
Virtual reality (VR) has been a long time coming. It has been through periods of high expectation – followed by disillusionment. In the 1990s VR involved headsets as large as a waste bin which had to be used in dark rooms, and because of the lack of computing power, caused nausea for many users within a few minutes. Only organisations with the budget of NASA could afford to use it for training.
Headsets are now smaller and lighter, computers are more powerful, and the cost of VR equipment has made it essential for serious computer gamers. The research organisation Gartner identified immersive experiences, which include virtual reality (VR) along with augmented reality (AR) and mixed reality (MR), as one of the top ten strategic technology trends for 2019. It predicts that VR systems will gain the ability to communicate with users across all human senses, including natural language conversation, shape-sensing, sound and touch (haptic feedback). By 2022, the report syggests that 70% of enterprises will be experimenting with immersive technologies, either for consumer or internal use.
Recent evidence suggests the nausea problem is no longer a barrier. Dr Natasha Taylor, Lead for Simulation at the Faculty of Health and Life Sciences at Coventry University explains that their research, which compared VR with conventional training, including traditional video, demonstrated that “VR is no more likely to provoke cybersickness symptoms than the other simulation methods. In addition, VR is reported as less tiring.”
For occupational safety and health, VR allows you to take learners to places that are too expensive (such as offshore) or too dangerous (such as the top of a wind turbine) to train in; it allows learners to fail safely; it can bring fires and explosions into a classroom. The British Safety Council has a VR risk assessment module as part of its classroom and distance learning courses, and Certified Skills use VR training within its classroom fire warden courses. Other organisations are developing in-house systems for training in both routine procedures and emergency operations, from selecting PPE and drilling holes to choosing harnesses and working at height, and from fire evacuation through fire-fighting to fire investigation.
Industry examples
Vodafone hired developer MakeReal to create immersive VR training for operatives around the dangers of work at height.
Steve Havvas, Virtual Reality Programme Manager at Anglian Water Services managed a project to use VR for driver training. He believes that VR training will save time and money “Using VR we can reduce the cost of driver training by 80% and we believe that retention rates with VR will up to 50% compare by traditional methods of training.”
John Kersey, HSEQ Business Partner for facilities company ISS has seen VR’s impact on training workshops within his organisation “When you are in a well-designed and well-rendered environment you feel that this is real and react accordingly. The experience breeds engagement and through sensory immersion – sights, sounds and feelings – it provokes an emotional response.”
Paul Speight of the Leicestershire Fire and Rescue Service was already convinced of the immersive benefits of 360 video for changing attitudes when he came across VR. “I could see it had a real potential for training in fire investigation – being able to bring realistic scenarios into the training room.”
Evidence
As far back as 1999 an article in Applied Ergonomics by John Wilson of Nottingham University’s Virtual Reality Applications Research Team reported that VR was “found more motivational, attractive and interesting by the trainees.”
But how much of this benefit might wear off once the technology becomes commonplace? Is there evidence that training with VR is effective?
In January 2019 the BBC broadcast a series where pairs of twins took part in experiments to test theories about diet, exercise and learning. The hosts of the series were twin brothers, doctors Chris and Xand Van Tulleken, and for one of the studies they volunteered themselves as guinea pigs. Xand spent an hour a day for a week learning to play baseball with a ball machine, while Chris spent the same time using VR. The advantage for Chris was that he didn’t tire as quickly during practice, as the weight of the controller was lighter than that of the bat. Xand meanwhile suffered blisters and muscle fatigue.
The neuroscientist consultant to the programme, Dr Jack Lewis claimed that VR training can “improve your brain’s ability to plot in space where you’d need to swing your arm to make contact with the ball.” The real-world trained Van Tulleken did do slightly better than the VR‑trained brother (possibly because of the muscle strength built up by a week’s practice), but the twins were surprised that their performances were “way closer” than they expected. They concluded “virtual reality can help.” This was a very small sample, there was no before measure to show that the twins started with a similar ability, and there wasn’t a “no training” control, so TV show results must be treated with extreme caution. However, results from peer-reviewed scientific studies can be equally open to interpretation.
What is VR
VR provides a representation of an environment that a user can interact with. As Wilson (see text) describes it “a participant can interact intuitively in real time with the environment or objects within it, and to some extent have a feeling of actually ‘being there’.” The visual representation is normally 3-D and can be computer generated or photographic. Stereo sound adds to the sense of realism. Some systems also provide haptic feedback – where the user can feel a virtual object through a haptic glove, or in limited cases through their entire body using a haptic suit. Most haptic feedback uses electrical muscle and nerve stimulation, but systems using ultrasound are also available. Although most VR involves a head-mounted display (HMD) which cuts out any view of the real-world, other VR systems are provided by Cave environments – where one or more people can be in a room surrounded by between three and six screens, with or without 3-D glasses. The least immersive VR is provided on a single wall screen, a desk-top screen or a tablet. Although lacking the immersion that comes from the HMD, it provides a way for those uncomfortable with HMDs to access the experience.
In entry level VR systems users have 3 degrees of freedom (3DoF) – they can look up and down, left and right, and turn around. The orientation of the headset is tracked, but the position of the user is not, so movement within a 3DoF environment is often achieved by “teleporting” – where the user appears to jump from one location to another. Add position tracking and you have 6DoF – the user’s movement in the virtual environment will simulate their movement in the real world.
The essential element to qualify as VR is that the user can interact with their environment and control objects within the environment. Where a video is 360 degree but linear, it is not VR.
Medical applications
Simulations using manikins have been used for many years in surgical and dental procedures. For specialist surgery, only parts of the body are needed – a head for a dentist, a torso for abdominal surgery. Adding VR to this mix provides clear benefits. Blood can be simulated if the surgeon nicks a blood vessel, or different levels of tooth decay can be superimposed on plastic teeth for a trainee dentist. In a review of the need for VR simulators in dental surgery, authors Roy, Bakr and George from the School of Dentistry and Oral Health at Griffith University in Australia point out that as well as immersion, VR has another benefit over physical simulators. “The ability of these simulators to store and replay students’ work further allows self-learning and assessment.”
The systems Roy and colleagues describe all have some physical components – with the dental students getting haptic feedback from their tools and the manikin. For ethical reasons the authors couldn’t compare the VR training with practising on real patients. Other VR studies similarly lack evidence that it is better than other approaches. A systematic review of VR simulators for spinal surgery in The Spine Journal in 2017 found 19 relevant studies, but concluded “higher-quality studies with patient-related outcome measures are needed.” A 2019 review in Radiography looked at ten studies of the use of VR in transvaginal ultrasound and could only say “although literature indicates that the simulator is valuable as a training/learning tool, there is insufficient evidence of measurable effects on clinical practice.”
One systematic review does have more positive findings. The review of laparoscopic surgery in the International Journal of Surgery in 2016 by Alaker, Wynn and Arulampalam found 31 studies of sufficient quality which compared VR training to no training and to other forms of training. A laparoscopic box has small holes through which surgical instruments are used, with a screen above the box showing the location of the instruments inside the box; not surprisingly, all methods of training were better than no training, and the VR training was “at least as good as box trainers”, and in some studies significantly better than other alternatives. The authors highlight one study that makes an important point about the type of VR you use “results showed that VR with haptic feedback … resulted in shorter operating time, less distance travelled and fewer unnecessary movements when compared to the VR non-haptic.” In other studies “lack of haptic feedback renders the VR trainer less effective than box trainers.”
Fire behaviours
A study by Çakiroğlu and Gökoğlu in 2019 in Computers & Education assessed the use of VR to train children aged 9 – 11. Children had five training cycles with the VR before they were assessed. The learning goals were for the children to get away from a fire and raise the alert with an adult. Each child was left alone in a room and “after a while, a smoke effect was given from a fog machine hidden inside the room and then the fire alarm was activated by using the sound systems inside the building.” The majority of the children trained with VR left immediately and found an adult to tell. A “real-world” comparison group with a dangerous hazard such as fire, particularly in relation to children’s behaviour, is not practical.
Later in 2019 I had the opportunity to visit an exciting project at Nottingham University, looking at how smell and heat could enhance VR used to train fire behaviours. See Feeling the heat.
Construction
Even without interaction and a HMD, immersion has benefits. A study reported by Perlman, Sacks & Barak in Safety Science in 2014 asked two groups of construction supervisors to identify and rate hazards in a construction environment. One group were provided with photographs and drawings of the site, while the other group wore 3-D glasses in a three-sided video cave. They could change their view within the construction site, but the route through the site was pre-defined. Even with these limitations, the group in the virtual environment identified more hazards correctly than the group who studied photographs and documents, and correctly assessed hazards caused by moving equipment as having a higher risk.
Emergency crew training
A 2015 paper in Computers in Human Behavior showed that when existing training is effective, VR might not provide any improvement. Two groups of German police were taught how to co-ordinate with helicopter crews. One group used VR, whilst the standard training involved a role-play activity. A third group had no training. The VR group were “more satisfied with their performance than the other groups” – but the standard group showed a better level of knowledge. Learning transfer was determined by assessing the behaviour of the police in a more complex role-play exercise. The conclusion was that “the virtual training was as good as the standard training” – as good, but no better. The decision is this case to use VR might therefore be dictated by how difficult, expensive or dangerous it is to organise the role-play training compared to VR training.
Remembering facts, changing perception
Because VR is about “being there” it is not as widely used for learning facts as it is for learning skills and behaviours. A recent study in Safety Science (Leder et al, 2019) set out to compare VR with PowerPoint as methods for delivering training about the safe use of a pillar drill. The study showed no significant differences in “recalled safety information and the knowledge of safety hazards” between the two groups. The authors conclude “the costly procedure of immersive virtual reality (VR) does not seem justified for safety training because the less costly PowerPoint procedure with vivid film scenes did not fare significantly worse.”
However, their “VR” was not interactive and had no haptic elements. The control group were presented with significant stills from the same video, with the same audio explanation. While there was no difference in the recall of facts, those in the moving video group perceived the risks of the pillar drill to be higher than the still image group – a finding that reflects the Perlman et al construction supervisor study.
Applying what we know
VR products differ in content, aim and immersion levels, so proving by experiment that it is more effective in any general way will always be difficult. Comparisons with the real world are often not possible – and hence the drive to create a VR solution. In measuring the benefits, the need to assess the transfer of learning to behaviour in the real-world rather than a score on a multi-choice test makes proof even more complex. Kersey sees an exciting future ahead where “natural language processing could be used to create a more natural interaction between the learner and digital ‘avatars’ within the virtual world.” As VR becomes more sophisticated, comparisons will become ever more difficult.
Rather than waiting for unequivocal evidence, start from the learning outcomes you want to achieve. If you want to teach facts you might be better with a self-paced, screen-based elearning package where the learner can repeat the content until they have absorbed all the facts. Alternatively, remove the need to memorise long procedures and the possibility of misremembering steps by using Augmented Reality (AR) to provide instructions on a heads-up display.
If you want to teach a physical skill, how easy, cheap and safe is it to teach that skill using the real equipment? If the skill requires expensive equipment or a dangerous environment, then VR might be a sensible route to follow. Be clear in your brief to the VR developer how the job must be done, and where the learner has options. The review of laparoscopic VR training demonstrated how important haptic feedback can be. It might be ok to select PPE using gaze direction or to select a risk rating by clicking a button when a cursor points at the right item. However, if you want to train people in the use of heavy equipment or in a process where fine motor control is needed, haptic feedback will be essential. The results of the Twinstitute baseball experiment also suggests that using VR for initial skill training could be a useful preparation for real-world training – for example, teaching the mechanics of driving or scaffold assembly before facing the hazards of other traffic or falls from height.
A properly immersive VR scenario might be worth the investment if your goal is to change attitudes. There is evidence that VR increases people’s estimation of a risk, and their ability to identify hazards. This might be relevant for hazard spotting activities or for encouraging rapid evacuation in emergency situations. However, check that you don’t inadvertently prioritise one set of hazards covered in the VR over more significant ones that are difficult to simulate.
Make sure that what the VR is teaching is correct. In VR I have been able to walk through corridors full of thick black smoke where the air at ground level is no clearer than that higher up; I have been able to put out improbably large fires with a standard water fire extinguisher, and I’ve been able to lift tonnes of rubble with the movement of a single finger.
Conclusion
The fact that VR is not always as effective as real-world training should not be a great surprise. If you have access to physical kit that you can train people with, if it is safe to train people in the real world, and you don’t need any detailed data collection you might stay with workplace training. VR can be a cost-effective option if you need to collect and review information on performance, or where real-world training environments are expensive or dangerous. VR might be a useful option to supplement traditional training to prevent the decay of skills and knowledge over time.
Whereas for some organisations elearning has become a standalone form of training, VR must remain part of a blended approach. As the dentists said “To a large extent the technology still may require considerable supervisor feedback at various steps of the procedure.” Havvas, while excited about the benefits of VR is clear “VR is complementing not replacing the conventional methods of training.”