This transcript has been edited for clarity.
Gayatri Acharya, MD: Greetings. I’m Dr Gayatri Acharya, cardiology fellow at Mayo Clinic. Today we will be discussing the role of virtual reality in medicine. I’m joined by my colleague, Dr Suraj Kapa, who specializes in this area. Welcome, Dr Kapa.
Suraj Kapa, MD: Thank you, Gayatri.
Virtual vs Augmented Reality
Acharya: What are augmented and virtual reality?
Kapa: People traditionally think of virtual reality as these expensive, bulky goggles you put on that allow you to be in an immersive environment and thereby play videogames or do other things that you see on television. But the truth of the matter is that these technologies have been around since the 1950s and 1960s, with people wanting to find ways of using machines to overlay data on their environments or to be able to interact with data better.
Nowadays, when we think of augmented reality and virtual reality, it’s really a continuum. Augmented reality is a way in which you can still perceive the world around you but perhaps use an iPhone (like with Pokémon GO) and project data into the surrounding environment. You still see the environment, but you have additional data within it. You can wear glasses, ranging from Google Glass to more expensive glasses, that allow you to use the power of a computer to project additional information. I could be sitting here looking at you and seeing a CT scan of your brain projected onto your body in real time, and interact with it and understand how it’s moving with your body. Or I can pull up a vital signs monitor around you to see exactly what your vitals are without having to pull up the computer. Augmented reality allows me to interact with you and your data more directly.
Virtual reality takes that a step further where, rather than having data portrayed into the surrounding environment, it puts you in an entirely different immersive environment. It’s like walking into the hospital where you are going to have your surgery beforehand and being able to see everything even if you had never been there before.
This is the range we think of in terms of augmented to virtual reality.
Acharya: How do you see this being applied to medicine and how we practice?
Kapa: The truth is, it’s already being applied. A number of groups have looked at virtual reality, these immersive environments, as ways of improving
depression, post-traumatic stress, and anxiety, reducing opioid needs for pain control, and reducing use of anesthesia for areas where you need moderate sedation or even deeper sedation.[1,2,3,4,5] They have shown that putting somebody in these immersive environments can actually help all of these situations.
I’ll give you one great example. We put a friend with severe Alzheimer’s disease in an immersive map environment and took her to where she was born and all the places she lived. It was fascinating because she was able to relay every single place—where she went to school, where she met her husband, and where her parents first picked her up after a dance recital. For a brief period of time during the immersion and afterward, she was completely lucid.
Education and Simulation
Kapa: If you look at the silos within which we can really advance this, the area of simulation and education is probably the one that is the most obvious. You can take an entire operative environment and project this into virtual reality, and allow any number of students anywhere in the world to be watching that surgery either in real time or afterward, to understand how the surgeon’s hands were moving. We all encountered this as medical students—trying to see exactly what the surgeon was doing with their hands but we had no idea what they were touching. What if we could stand in the surgeon’s position and look directly down and see exactly what they are doing, what they are touching, and how their hands are moving?
This is already being done. A group in London have been transmitting their surgeries in real time to hundreds of thousands of students worldwide, allowing them to get that exposure no matter where they are, no matter what their medical school does or where it’s located. We can advance education.
With simulation, like we talked about before, you could walk into an environment beforehand that you might need to work within, but right now the only way to do that is to be there. That is one silo.
Kapa: A second silo is data integration. Within medicine, especially procedural characteristics, we get these data beforehand, such as imaging tests, vitals, or electrocardiographic monitors. Then we walk into the lab or operating room, throw on our sterile scrubs, and have to remember what we looked at. Wouldn’t it be so much better if we could pull up data in that sterile environment and see how it’s integrating with what we’re seeing in real time?
Kapa: The third silo, which I think is probably the most important or potential one, is how do we educate our patients better? How do we integrate them into the space of their own clinical care? How do we take care of them at home when they are located hundreds of miles away from the nearest hospital, and provide them with physical rehabilitation leveraged through virtual avatars of physical therapists who can train them through needed exercises without having to find a rehab center?
These are three big areas within which I see these technologies evolving.
Application of Virtual Reality
Acharya: These all sound great, but how far away from this technology are we? How are we going to implement this at Mayo Clinic?
Kapa: We are getting there, but we also have a ways to go because we are evolving as the technology evolves. Where we stand right now, a pipeline is being built, where every CT scan and every MRI scan is automatically pipelined into a 3D compilation system that can immediately segment those data into a 3D image that we can work with and cut through any which way we want within an immersive 3D environment. For example, we can understand the complex congenital heart and how things relate to each other.
We’re working in the area of ICD lead extraction to find where the lead is actually adhered to tissue, because we can alter the parameters of the tissue density much more than we can in a standard 2D CT scan that we watch on a computer screen.
In addition, we’re working to digitize every procedure room in the hospital. People who will be training and working within it can work with the actual equipment in that room before they ever physically walk into it so they can understand it—for example, radiation safety for a specific fluoroscopic machine.
The third way—and this is both futurist and now thinking—is integrating cameras into our operating rooms where we’ll be able to capture all the 3D data of what is happening inside. One of the biggest developments in medicine and modern medicine was the surgical checklist. The impact that had on safety and outcomes was huge. But we still need people writing all the things down, checking off all the boxes. They have to have their eyes everywhere. What if we can do that by creating a 3D reconstructed environment that allows a computer to automatically check everything off, to ensure more accuracy in more real time with a reduced number of resources?
These are some areas where we’re already working, but the eventual paradigm as these devices become more miniaturized is to implement them in our medical schools, our daily practice, and how we work with our patients from the time they walk into the waiting area through the time they come into your office, wondering what you’re going to say ahead of time.
Those are three big areas where we’re working already, but there is a vast area of potential that might evolve as we get different, newer, or more evolved devices.
Acharya: I think those are things that we’ll start to recognize as we start to use the technology. As a trainee, it would have been extremely helpful to have some familiarity with the environment you are walking into, or to have a more advanced visual aid that specifically addresses the patient’s anatomy, instead of my crude drawings that may not fully be applicable to the patient or that may be difficult to understand.
In what other areas do you see us being able to use this technology in medicine?
Kapa: One area where we might be able to use this technology is to advance our relationships, and not just within the clinic, the people we’re working with, or with the data that we’re working with. When you think about how Mayo Clinic first started, the Mayo brothers were not just about creating the best hospital within the cornfields of Minnesota. Their paradigm was one of travel and bringing together information, of teaching others as well as bringing back what others can teach them. This holistic principle, which underlay them traveling in a ship that ran up and down the Mississippi River to deliver care, never accounted for these vast computational technologies that might allow us to talk to somebody who is doing a surgery on the other side of the world and actually be in their operating room, telling them what they might be able to do differently. Or being able to login to a patient’s room in a different state or in a rural area and be able to tell them, “I can listen to you and I can talk to you face-to-face and you don’t have to travel.”
These technologies will allow democratization of medical access, and I see that as being the future of these devices being integrated into our practice.
Acharya: It’s all very exciting, and I can’t wait to see where we take this technology. Dr Kapa, thanks for these very important insights, and thank you, the audience, for joining us on theheart.org | Medscape Cardiology.
Follow Medscape on Facebook, Twitter, Instagram, and YouTube