Introduction:
Imagine the year 2030… The internet has evolved into what we now call as the Metaverse. In this new digital realm, you navigate through a highly sophisticated virtual world as new digital representations of your avatars – digital humans. Healthcare professionals’ access three-dimensional (3D) images of patients’ scans, tele-collaborate with multidisciplinary teams, use patients’ digital twins to discuss diagnosis and treatment plans, and document all their medical notes in the patients’ metaverse health records.
The above scenario may seem right out of a sci-fi movie, but this digital shift is not as distant as it may seem, especially as the world increasingly embraces digital transformations in the post-COVID era. This article aims to provide readers with a fundamental understanding of immersive technologies, such as extended reality (XR) and the emerging metaverse ecosystem. It also delves into the current trends and applications in healthcare.
Understanding the Reality-Virtuality (RV) Continuum:
Healthcare professionals are already exploring various immersive technologies to enhance clinical practice and education. However, there are differences between virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR), which if understood, can facilitate better communication between healthcare professionals and developers.
To grasp these nuances, consider the Reality-Virtuality (RV) continuum (Figure 1). One end of this spectrum represents the physical or real world, where individuals interact with tangible objects within a real-world environment. On the opposite end of the spectrum is VR, where users engage with virtual elements within a simulated or digital environment. Initially concerned mainly with visual displays, it is now updated with further classifications based on its virtual environments.1 The first type is an external virtual environment (EVE), where users wear VR headsets/devices and interact with virtual objects while immersed in a virtual environment. However, these devices limit the user’s immersion, since they are physical objects used in the real world. In contrast, the second type, known as a “Matrix-like” virtual environment (MVE), provides a higher level of immersion because it not only stimulates the users’ exteroceptive senses (e.g., sight, smell, taste, touch, and hearing), but also their interoceptive senses (e.g., proprioception). The MVE system is aware of, and can respond to changes in real-world environments.
Between the physical world (reality) and MVEs lies a spectrum that blends real-world and virtual elements, allowing users to perceive both simultaneously. This blended experience is known as MR, which comprises of AR and augmented virtuality (AV) (Figure 1). In AR, virtual elements are overlaid in the physical environment, offering limited interaction with virtual objects. On the other hand, AV involves integrating real-world objects into a virtual environment, enabling users to interact with physical objects that appear in a virtual space.
Extended reality (XR) encompasses all these different forms of reality, excluding the physical world. It combines the real and virtual environments with human-machine interactions, facilitated by computer technologies and wearables. In XR systems, users typically engage with and perform tasks in real-and-virtual environments through various sensors and input devices. The data generated is captured and can be analyzed using artificial intelligence (AI) algorithms before being relayed back to the user through output devices.
Figure 1. The reality-virtuality (RV) continuum.
Application Trends of Immersive Technologies in Healthcare:
This section explores a non-exhaustive list of immersive technological trends in healthcare. These examples are for educational purposes and intended to increase awareness of the innovations in this field. They are not meant to replace professional medical advice or consultations with healthcare professionals.
(i) VR Rehabilitation:
VR rehabilitation is increasingly used as a means to enhance motivation, engagement and adherence to traditional movement-based therapies, particularly for home-based rehabilitation. This mode of therapy has shown some effectiveness in conditions such as stroke and cerebral palsy, as well as for pain control. For example, a pragmatic randomized trial in Australia demonstrated clinically significant improvements in mobility among participants who used VR applications in addition to conventional rehabilitation. Another small-scaled randomized-controlled study involving patients with cerebral palsy in Turkey showed significant improvement in motor function and mobility in the intervention group.
(ii) XR Telehealth:
Telehealth gained popularity during the COVID-19 pandemic, with companies exploring the combination of XR and telehealth to provide safe and immersive healthcare experiences. For example, XRHealth created a VR Telehealth Virtual Clinic, offering personalized care plans and gamified exercises for patients to perform at home while staying connected to their therapists through video calls and in-app messaging services. Other applications included AR systems for remote consultations and ventilator management for COVID-positive patients, VR systems for training in trauma and emergency medicine, and VR tele-mental health.
(iii) Virtual Reality Exposure Therapy (VRET):
VRET is an approach that employs VR technology for graded exposure therapy. It has been used for the treatment of mental health disorders, including specific phobias, and post-traumatic stress disorder (PTSD). VRET immerses individuals in a virtual environment, allowing them to confront their phobias or trauma stimuli in a controlled and safe manner. It has demonstrated effectiveness in treating conditions such as acrophobia, fear of flying, dental phobia, agoraphobia, and panic disorder, among others.
(iv) VR in Oncology:
VR has been used as a distraction technique to alleviate pain and anxiety in patients with cancer, such as during chemotherapy sessions and in palliative and supportive care. Studies have shown that VR significantly reduces pain levels, particularly when using high-fidelity VR technology, as it helps in distraction by immersing patients more deeply into the virtual environments.
(v) XR-based Surgery:
XR applications in the surgical field tend to support preoperative planning, intraoperative guidance, and surgical education/training. Benefits include improved understanding of anatomical structures, enhanced visualization of anomalies, improved surgical planning and execution, and increased surgical precision. XR-based surgical training also improves surgeon confidence, procedural time to completion, post-intervention scores on procedural checklists, and greater implant placement accuracy.17
(vi) AR-based Digital Operations:
AR can also streamline healthcare operations. Companies like Augmedix use AR and natural language processing to develop digital scribes for clinicians, allowing them to document medical notes in real-time during patient consultations, thus cutting down on administrative workload. Another example is AccuVein, which uses projection AR technology to visualize patients’ vein vasculature, aiding in blood draws.
(vii) Digital Education in Healthcare:
VR technologies have been used to train medical students in soft skills such as empathy and health communication. The Royal College of Surgeons in Ireland has also used VR simulations for clinician training on managing emergencies. Additionally, various institutions have employed XR for various forms of training, such as septic shock management, cardiopulmonary resuscitation, medication safety, and certain procedural skills.
From XR to the Metaverse:
Many tech companies and organizations have embraced the metaverse following Mark Zuckerberg’s vision in 2021, believing that it will bridge the gap between the physical and digital worlds, and provide users with unprecedented immersive experiences. Gartner predicts that in the next few years, 25% of people will spend at least an hour daily engaged in metaverse activities, including work and entertainment. Comprising of 4 main types (AR, lifelogging, mirror worlds, and virtual worlds), each will offer unique opportunities for interaction and immersion.
While there is a lack of published literature on the potential of the metaverse in healthcare, it is believed to have significant applications in health prevention, treatment, education and research. The emerging concept of Health 4.0 integrates technologies like XR, the Internet-of-Medical-Things (IoMT), cyber-physical systems, cloud computing, big data analytics, AI, and blockchain in healthcare. Virtual care delivery through clinician avatars in virtual clinics, telehealth services, and remote patient monitoring has been envisioned. In healthcare education, the metaverse can offer a more immersive and creative platform for sharing knowledge. In fact, it has already been explored for the education of healthcare professionals, such as for continuing professional education, and broadcasting surgeries to online audiences in real-time.
Generative AI and the Metaverse – To Infinity and Beyond:
Advancements in AI, particularly generative AI, are expected to enhance the metaverse’s immersive capabilities and user engagement. The creation of dynamic, original content through generative AI (e.g., text, images, videos, music, 3D objects/assets, avatars) can provide users with unique and immersive experiences. Social connections and collaborations in the metaverse, driven by large language models (LLMs), are expected to accelerate. The capabilities of generative AI in terms of producing dynamic video content, virtual actors, video synthesis, avatar animations and multi-sensory interactions, can potentially make user avatars more interactive and realistic. Users may also engage in more lifelike conversations with non-player characters in the AI-driven metaverse. Furthermore, technological enablers such as patient digital twins, secure data sharing, enhanced regulatory frameworks, and virtualized medical interventions based on patient-specific simulations in the metaverse can potentially revolutionize healthcare practices.
While the younger generations, referred to as “Generation Meta”, are embracing digital life through social media, gaming, and the metaverse, the healthcare industry is also gradually recognizing the potential of virtual care. The rapid growth of telehealth during the COVID-19 pandemic suggests that virtual care is on the rise. Initial efforts to explore the metaverse for healthcare education and practice imply that the healthcare metaverse may become a reality, even though mainstream adoption may take time. As we move forward, healthcare practitioners will need to be both passionate and interdisciplinary, possessing both clinical knowledge and technological skills to navigate the immersive technologies of this emerging digital frontier.
