As a medical training and professional development tool, virtual reality is still in its infancy. So what does the future hold?
Imagine it’s 10 years in the future. You’re a medical student studying anatomy, but there are no weighty tomes, dense with text and two-dimensional cross-sections of body parts. Instead, an anatomy class has more in common with a trip to an art gallery, with you wandering around and inspecting bodies as a guide narrates the journey via a headset.
But it’s even better than an art gallery. As you walk around a body you can choose whether you want to hide the skin, zoom into the skull, or pull out the thorax for a closer inspection. It may sound a little morbid, but rather than a cadaver, this is a collection of pixels beamed to your virtual reality headset.
Australia is leading the way in this cutting edge virtual reality which could drastically change how many doctors are trained, with some proponents claiming the technology could eventually replace the need for cadavers.
Out of the Gold Coast comes the 3D Organon VR Anatomy, the first fully-featured virtual reality anatomy atlas, which is now being used in universities across the world. You can see why, given the program includes more than 4000 different anatomical structures with labels and descriptions that students can explore by hiding or fading organs, veins and bones they don’t want to see, and being able to fully rotate the ones they do.
Professor Michelle Moscova, senior lecturer in anatomy at University of NSW, allowed this writer to experience the version she and her colleagues have been trialling with students in the university.
Sitting in front of a high-powered laptop I placed on the thick set of goggles and earphones that are the headset. Suddenly I’m in a plain white room with a large, skinless human standing in front of me. Turning my head left and right reveals a desk and chairs in one corner and a surgery table and equipment in another.
But that’s just the background. When I keep my eyes focused on the body and coordinate, or try to, with my mouse scroller, I can bring myself close to the torso. Clicking at bits of the body makes them disappear to reveal more structures underneath, or I can pull up a description of the structure to find out more. I click on one and a thin white strip comes up next to it. “Pectoralis major, L,” it reads. I click another button, “pectoralis major” a monotone voice tells me.
Professor Moscova and others can see what I’m seeing via the laptop screen, and I can hear their conversations, but the world in front of me is fully immersive.
Back on the menu screen there’s a choice of body parts to explore. Do I want to focus on the cardiovascular system, or the sensory or reproductive one? Or maybe I’ll choose the nervous system, and from there narrow it down to the cranial nerves, meninges or ventricles. The complicated curves of the vestibular are blues and greens and reds to make it easier to demarcate the different parts.
For Professor Moscova, the sheer importance of virtual reality may be on par with Andreas Vesalius’ set of anatomy books, De Humani Corporis Fabrica.
“The last thing this exciting probably happened with Fabrica, which is the first anatomical comprehensive text, which was published centuries ago,” she says. “That really revolutionised anatomy, because prior to that, we relied on dissection demonstrations and sketches. That was the first book that really collated the knowledge in one place, and substantially improved anatomy learning.
“But I think virtual reality makes it more accessible for students, who cannot always be on campus.”
There are many students who study by distance, or who study on campus and then have to go to placement to remote hospitals, which may not have cadaver facilities when they might need them, Professor Moscova says.
“I think for those type of students, virtual reality can make a very big difference.”
Cadavers are the gold standard, but unfortunately it can be challenging to recruit volunteers, and even for students in cities, the cadaver laboratories can’t be available 24 hours a day, seven days a week.
The possibilities of expanding access and upgrading the learning experience are intriguing, but the reality is that with the headset, expensive laptop, and the program itself, universities may still be looking at spending thousands of dollars per student on the new technology.
So Professor Moscova wanted to know if using virtual reality actually translated into better learning for students.
“How do you test the outcomes?” she asks. “It’s all well and good if students are going ‘wow red, wow white’, but that doesn’t mean they are learning the content any better than they were before.
“The issue with all new technology is that we actually don’t know how to use it effectively, so we have to experiment to see what works best,” Professor Moscova says.
Research into the best use of virtual and augmented reality for medical training is thin on the ground. For Professor Moscova’s team, they plan to undertake a more controlled study comparing the tool with more traditional methods for learning anatomy over this summer break.
Bond University has been a major developer of virtual reality in the medical and allied-health realm, and a team there tested out virtual reality and augmented reality against standard, tablet-style anatomy lessons.
The Bond University team, which includes Organon creator, Assistant Professor Athanasios Raikos, found performance on an anatomy quiz was similar regardless of whether their 59 students were randomly allocated to virtual reality, augmented reality or a 3D tablet-based module.
But where virtual and augmented reality offered greater promise was in the higher levels of immersion and engagement the students experienced, the study published earlier this year, found.
While this is encouraging, it is not what you want if you’re considering dropping thousands of dollars fitting out an entire class out with this equipment. Medical students have multiple competing priorities, so as fun as virtual reality learning might be, spending hours on it might be taking time away from potentially more useful study.
STUDENT TRAINING
To test the technology, Professor Moscova took a group of 30 pre-med students and split them into two groups. The first group got to play around with the virtual reality anatomy atlas first and the other group went and got their hands dirty with a cadaver.
The first group were given a clinical problem to solve while exploring the virtual body, such as being asked to find nerves or blood vessels that could be damaged if a specific bone was fractured and a structure displaced.
Then, halfway through the three-hour tutorial, the two groups swapped.
They found that around two in every three students preferred to start in the virtual reality lab first before going into the cadaver lab.
“Even though we asked them to solve problems that required some knowledge of anatomy, they preferred to do that,” Professor Moscova says. “[The structures] are nicely coloured in the virtual reality application, you can switch them on and off and you have the option of removing structures you don’t want to see.”
What the students found was that it made it a lot easier for them to interact with the cadavers when the time came.
The tutors who oversaw the cadaver session reported the same thing, saying the students who used virtual reality first were far more self-directed and independent when it was their turn with the cadavers.
In this situation, virtual reality was a valuable “pre-learning exercise”, Professor Moscova says.
For Professor Raikos, the fact virtual reality is so solitary is a huge benefit of the method.
Compared with other classroom activities, which include classmates, phones and distractions, once you put on a virtual reality headset it’s just you alone in the world, he says.
“When you attend a lecture, you listen to a lot of information but after a few months you forget 90% of it,” he says. “When you interact yourself, by holding something or moving something or by solving a problem, this is how you retain knowledge.”
Because the HTC Vive virtual reality technology is designed as a walking experience where the user can walk around the model, grab bone and put it somewhere in three-dimensional space for example, it becomes a kinaesthetic interaction, he says.
“It’s much better than reading from just a book or a webpage. A really important advantage is that by doing a module in virtual reality you are immersed there, because you’re wearing a mask, if you put those headphones on then you are there and the distractors don’t exist,” Professor Raikos says.
But the flip side of that, Professor Moscova explains, is that it’s bad for group learning, which is pivotal to much of medicine. The students are on their own, not with a teacher, not seeing what the rest of the class is. Some problems are much better when they’re solved in teams, she says.
“That is where virtual reality can be challenging. Although they had all this technology, and they loved it, cadavers were preferred far, far more,” Professor Moscova says.
It’s not that virtual reality was lacking though, rather that cadavers are simply more authentic, she says.
While one drawcard of virtual reality is the bright, different colours for different areas, allowing students to become more adept at picking out different structures and their relationships, Professor Moscova says it is also integral to training to have the realism that a cadaver provides, with all its indistinct colouring and messiness.
These virtual reality applications are based on an ideal human, but real medicine isn’t about dealing with an ideal human, she says.
“There’s not just one normal, there are a thousand normals,” she says.
Nevertheless, it is important not to discount the importance of the student experience, Professor Moscova says. Virtual reality is so exciting and obviously appealing to people that it motivates people to learn, overcoming a major barrier in the education process.
Another potential worry, however, is that around one in three of Professor Moscova’s students reported side effects, such as nausea, from the virtual reality experience.
When I popped the headset on and jumped into the Organon world, she recommended I slow down instead of zooming around the skeleton, pinging up and down vertebrae like I was in in a skyscraper lift. After taking her advice, my early signs of nausea quickly receded.
On top of that, Professor Moscova found a few unexpected cases of blurred vision, possibly a sign of undiagnosed long-sightedness. And for those who already wear glasses, the current headsets make it quite uncomfortable to wear glasses underneath – although virtual reality developers are currently working on headset-based solutions to that issue.
Despite quite a high number of side effects from the devices, Professor Moscova says she was surprised to find that almost all of the students said they would use it again.
Price is a challenge, but like all new technology it will inevitably go down, she says. For individuals who already own Oculus Rift or HTC Vive virtual reality technology, the award-winning Organon program only costs $30.
Another member of the virtual reality lab at Bond University team is Associate Professor Allan Stirling, a surgeon and professor of clinical anatomy. Professor Stirling points to augmented reality as another exciting development in this area.
The team at Bond has created augmented reality programs for tablets that can be used for medical training.
“We have a special cube that has colours and shapes on it that the camera recognises, and it shows you a rendering on the screen of the anatomical object,” he says. “You can then physically move that cube with your hands and it will rotate on the screen and you can strip away the layers.”
Narration accompanies the experience, so that students can listen to instructions through their headphones.
“The learning is paced at their own level, as opposed to a lecture, where if you don’t get it you can stop play and rewind and listen again. When we mention a certain structure, it’ll glow or highlight on our model they’re looking at,” Professor Stirling says.
Some students prefer that because they initially don’t enjoy being wholly immersed in the program and not being able to see the surrounding world.
Additionally, some people cannot conceptualise things in three dimensions as well as others, so a tool such as this helps them to learn and allows them to do it at their own pace, he says.
Nevertheless, he echoes Professor Moscova’s point that their students also saw the role of virtual reality as an adjunct, rather than a replacement, to traditional teaching methods.
PROFESSIONAL DEVELOPMENT
The potential of virtual reality doesn’t end in the training phase of a doctor’s career though.
Professor Stirling says that using virtual reality helps his colleagues visualise and explore different “lumps and bumps” on the spine in preparation for an operation. This kind of technology could also be useful during the surgery itself.
A key player in the augmented-reality field right now is technology giant Microsoft with its HoloLens, which is being tested for surgeons to augment their field of vision for some of the pre-operative data.
For example, one program has a special camera that can pick up which angle the surgeon is putting the screw in at and the program will superimpose the patient’s radiological data onto the spine, or depict lines to guide the surgeon so that the screw goes in the right angle, the right depth and in the right place, Professor Stirling says.
Providing patient data through a lens helps save doctors from having to refer back to notes, and it operates via gestures so it’s handsfree.
This kind of technology is known as medical mixed reality, because it layers computer-generated images on top of the real world, much like Snapchat filters and Pokémon Go work.
Professor Stirling says we could see this kind of technology in the education sphere within the next few years, but there is typically a lag time between commercial rollout and when the tertiary setting starts using it.
For radiologists too, virtual reality holds great potential in being able to transform two-dimensional slides into a three-dimensional and more real-life visual of the test results. It would also allow people to view the images in high fidelity while working remotely or on the move.
Even more surprisingly, virtual reality may be utilised as a way of building empathy and understanding in younger doctors.
“It’s not always that they have the life experience that somebody with 20 years of medical practice might have, and there are certain topics that can present as very difficult for students,” Professor Moscova says.
“For example, what is it like to be a person who has a particular disability? What is it like to be a mental-health patient who is experiencing PTSD or a psychotic episode? Those kind of things you can’t simulate, but I think with virtual-reality applications there is the potential to develop technology to allow them to be in the shoes of someone who is actually experiencing this kind of thing.”
Alzheimer’s Australia won multiple award for its Virtual Dementia Experience launched in 2013, which it describes as “an immersive, interactive virtual-reality experience that invades the senses and takes people into the world of a person living with dementia, simulating thoughts, fears and challenges”.
Similarly, the eating disorder patient group, The Butterfly Foundation, helped create a virtual-reality film with the aim of letting members of the broader community see the world through the eyes of someone suffering from an eating disorder.
Patients themselves could benefit from learning about their own illness through Organon’s virtual reality anatomy experience.
But there is something special about the first time somebody sees a cadaver, or holds a brain in their hands, Professor Stirling says.
“We hear about cadavers going away,” Professor Moscova says. “But we’ve heard that before.”
“[These new technologies] add a lot of potential, they really expand and push the boundaries for us as educators, but they don’t do away with the tools that have served us really well for the last 400 years.”
