The Future of VR

A Brief History of VR

Haptic Immersion vs Full Dive

The Ready Player duology, consisting of Ready Player One (2011) and Ready Player Two (2020), is one of the most mainstream works that discusses VR technology in great depth. The novels hold an exciting narrative told by Wade Watts, a teen living in the year 2045 on a dystopian future earth. He and many others use the OASIS, a virtually simulated utopia to escape from a boring, impoverished life and the declining state of the planet. Both in the novel and in real life, the OASIS is used to play games, watch videos, hang out with friends, attend class, go to work, and so much more. The first novel focuses mainly on haptic immersion technology, while the second novel shifts to full dive technology. As the books are set in a not-so-distant future that centers on the emerging technology we are beginning to see, they provide an interesting framework from which to consider the potential impact of this technology.

Let’s talk more in-depth about two main categories for this technology:

Haptic Immersion

Haptic immersion is the branch of VR that relies heavily on using haptic sensory technology to give the user a physical feeling such as a vibration while experiencing VR. By using things such as haptic gloves, and in some cases, haptic vests or suits, this branch of VR allows a user to put on a headset and experience VR with more realism. Haptic immersion also relies heavily on the physical motions a player makes–sometimes incorporating a treadmill or slide-pad (slippery surface to run on with low-traction shoes).

The haptic sensory technology allows the user to feel a new dimension of sensory input from the games they are playing. Some simple applications of haptics might be allowing the user to feel as if they are actually holding objects or allowing more player interaction, such as tapping another avatar on the shoulder. Imagine you are playing a first-person shooter game and that you are attacked by an enemy from behind. If you are playing on a computer, the damage input would typically be shown around the center screen crosshair with the direction of the damage indicated with an arrow or line. By playing with haptic immersion, however, you would be able to feel the exact spot that the damage is dealt, giving you the ability to react more quickly to the attacks and providing for a more immersive experience.

The first description of haptic immersion we see in Ready Player One is most similar to the technologies we currently have available today.

The wireless one-size-fits-all OASIS visor was slightly larger than a pair of sunglasses. It used harmless low-powered lasers to draw the stunningly real environment of the OASIS right onto its wearer's retinas, completely immersing their entire field of vision in the online world. The visor was light-years ahead of the clunky virtual-reality goggles available prior to that time, and it represented a paradigm shift in virtual-reality technology-as did the lightweight OASIS haptic gloves, which allowed users to directly control the hands of their avatar and to interact with their simulated environment as if they were actually inside it. When you picked up objects, opened doors, or operated vehicles, the haptic gloves made you feel these nonexistent objects and surfaces as if they were really right there in front of you. The gloves let you, as the television ads put it, ‘reach in and touch the OASIS.’ Working together, the visor and the gloves made entering the OASIS an experience unlike anything else available, and once people got a taste of it, there was no going back. -Ernest Cline, Ready Player One, Page 58

 Then, we have descriptions of things that still lie in our future.

I spent the majority of my time in my Shaptic Technologies HC5000 fully adjustable haptic chair. It was suspended by two jointed robotic arms anchored to my apartment's walls and ceiling. These arms could rotate the chair on all four axes, so when I was strapped in to it, the unit could flip, spin, or shake my body to create the sensation that I was falling, flying, or sitting behind the wheel of a nuclear-powered rocket sled hurtling at Mach 2 through a canyon on the fourth moon of Altair VI. The chair worked in conjunction with my Shaptic Bootsuit, a full- body haptic feedback suit […] The outside of the suit was covered with an elaborate exoskeleton, a network of artificial tendons and joints that could both sense and inhibit my movements. Built into the inside of the suit was a weblike network of miniature actuators that made contact with my skin every few centimeters. These could be activated in small or large groups for the purpose of tactile simulation--to make my skin feel things that weren't really there. They could convincingly simulate the sensation of a tap on the shoulder, a kick to the shin, or a gunshot in the chest. (Built-in safety software prevented my rig from actually causing me any physical harm, so a simulated gunshot actually felt more like a weak punch.) I had an identical backup suit hanging in the MoshWash cleaning unit in the corner of the room. These two haptic suits made up my entire wardrobe. My old street clothes were buried somewhere in the closet, collecting dust. On my hands, I wore a pair of state-of-the-art Okagami IdleHands haptic datagloves. Special tactile feedback pads covered both palms, allowing the gloves to create the illusion that I was touching objects and surfaces that didn't actually exist. -Ernest Cline, Ready Player One, Pages 191-192

As mentioned before, there is nothing quite like this on the market as of yet, and a lot of what comes close is not readily available for mass consumption due to either high prices or the production is mainly focused on high-end business usage. The cost it takes to produce high-end haptic technology puts it way out of the price range of the average consumer. One such example of this comes from the company haptx.

While we don’t yet have any high-end haptic chairs that are suspended from the ceiling like in Ready Player One, we do have some basic haptic suits that are on the market. The current main provider of haptic suits is bHaptics who offer a range of haptic technologies like vests with varying feedback points, haptic sleeves, haptics for VR HMDs, and haptics for hands (not gloves) and feet. And they’re pretty pricey. The “cheapest” haptic vest with only 16 feedback points comes in at $299, and the most expensive with PC compatibility and 40 feedback points is at $549. If you’re interested in the full experience, you’re looking at a price of about $1,400+, and it’s not even a proper full haptic suit.

Another company, TESLASUIT (not affiliated with Tesla), has developed a full-body haptic suit, though it is only for industry use. Considering the cost and inaccessibility of these technologies, it’s not hard to see why discussions about haptics haven’t broken the internet yet.

Haptic immersion uses these technologies in coordination with VR headsets to bring users a more physically engaging VR experience. Some future potential applications could include military training or physical therapy. Military training in VR might allow soldiers to have exposure to more dangerous scenarios without the worry of facing actual danger. The haptic technologies would allow for them to feel the consequences of their actions without the setback of having a lasting injury. Physical therapy in VR would allow for a more engaging experience, allowing, for example, the patient to choose a fun environment to transport themselves away from the doctor’s office for the duration of their visit. If the haptic suits could be combined with some form of hard robotic exoskeleton, it could allow for those doing physical therapy to have a more exciting and engaging experience. The addition of a robotic exoskeleton could allow for stretches that a patient may be unable to complete on their own, helping them on their journey to a potentially quicker recovery. The possible applications for haptic immersion give this branch of VR a future full of technical applications that may enhance the way we view physical activity and training.

Full Dive

Full dive is a branch of VR that has had a lot more media exposure. At its core, full dive is VR for your mind. It relies primarily on a brain-computer interface (BCI) to allow a user to control the virtual world with their thoughts. This technology is currently undergoing research, and there have been no real instances of it being a reality as of yet. In theory, a true full dive would allow a user to put on a VR headset with BCI capabilities, and the headset would intercept their brain signals, allowing the user to become their virtual avatar while in the simulation. Imagine the possibilities that this technology could present: reduced travel costs, more immersive educational experiences, being able to visit any place or any time period, or working a job you would never be able to do otherwise. Ernest Cline engages with this concept in his book Ready Player Two:

The device had a segmented central spine that appeared to stretch from a wearer's forehead to the nape of their neck, with a row of ten C-shaped metal bands attached to it. Each band was comprised of jointed, retractable segments, and each segment had a row of circular sensor pads on its underside. This made the whole sensor array adjustable, so that it could fit around heads of all shapes and sizes. A long fiber-optic cable stretched from the base of the headset, with a standard OASIS console plug at the end of it. […] ‘The device you now hold in your hands is an OASIS Neural Interface, or ONI.’ He pronounced it Oh-En-Eye. ‘It is the world's first fully functional noninvasive brain-computer interface. It allows an OASIS user to see, hear, smell, taste, and feel their avatar's virtual environment, via signals transmitted directly into their cerebral cortex. The headset's sensor array also monitors and interprets its wearer's brain activity, allowing them to control their OASIS avatar just as they do their physical body--simply by thinking about it' -Ernest Cline, Ready Player Two, Pages 5-6

The term “full dive” was coined in the 2016 anime franchise, Sword Art Online, though the concept was brought to mass media a lot earlier in The Matrix, which was released in 1999. The Matrix revolves around the main character, Neo learning that the entire world is a simulation created to keep humans complacent while AI harvests their bodies for energy. In Sword Art Online, the main character is trapped in a virtual MMO (massively multiplayer online game) by the creator of the device along with all the other players. The creator had trapped the players in the game by removing the log-out option. The only way one can escape the simulation is to beat the game.

Its telescoping bands retracted automatically, pressing the array of sensor and transmitter pads mounted on them firmly against the unique contours of my cranium. Then its metal joints tightened up and the whole spiderlike device locked itself onto my skull so that its pads couldn't be jostled or removed while the device was interfacing with my brain. According to the ONI documentation, forcibly removing the headset while it was in operation could severely damage the wearer's brain and/or leave them in a permanent coma. So the titanium-reinforced safety bands made certain this couldn't happen. I found this little detail comforting instead of unsettling. Riding in an automobile was risky, too, if you didn't wear your seatbelt … The ONI documentation also noted that a sudden power loss to the headset could also cause potential harm to the wearer's brain, which was why it had an internal backup battery that could power the device long enough to complete an emergency logout sequence and safely awaken the wearer from the artificially induced sleeplike state it placed them in while the headset was in use. -Ernest Cline, Ready Player Two, Page 9

It is this aspect of the technology that Sword Art Online, The Matrix, and Ready Player Two all highlight over the course of their respective narratives, and it is these things that make some people (myself included) skeptical about the progression towards this technology. These narratives show the dangers that this technology could pose if some maniacal creator or AI might take advantage of the BCI to trap their users inside the simulation.

The potential ethical issues that arise with any type of VR, especially with BCI-enhanced VR, are innumerable. When approaching full dive and any technology really, both creators and users must keep in mind the potential negative outcomes that may arise from the creation of such a technology. The applications for full dive in conjunction with accessibility research is presented at the beginning of Ready Player Two.

A few months after GSS launched the OASIS, Halliday set up an R&D division at the company called the Accessibility Research Lab. Ostensibly, its mission had been to create a line of neuroprosthetic hardware that would allow people with severe physical disabilities to use the OASIS more easily. Halliday hired the best and brightest minds in the field of neuroscience to staff the ARL, then he gave them all the funding they would ever need to conduct their research. The ARI's work over the next few decades was certainly no secret. To the contrary, their breakthroughs had created a new line of medical implants that became widely used. I’d read about several of them in my high school textbooks. First, they developed a new type of cochlear implant that- for those who chose to use it--allowed the hearing impaired to perceive sound with perfect clarity, both in the real world and inside the OASIS. A few years later, they unveiled a new retinal implant that allowed any blind people who wished to be sighted to "see" perfectly inside the OASIS. And by linking two head-mounted mini cameras to the same implant, their real-world sight could be restored as well. The ARI's next invention was a brain implant that allowed paraplegics to control the movements of their OASIS avatar simply by thinking about it. It worked in conjunction with a separate implant that allowed them to feel simulated sensory input. And the very same implants gave these individuals the ability to regain control of their lower extremities while restoring their sense of touch. They also allowed amputees to control robotic replacement limbs, and to receive sensory input through them as well. To accomplish this, the researchers devised a method of "recording" the sensory information transmitted to the human brain by the nervous system in reaction to all manner of external stimuli, then compiled these assets into a massive digital library of sensations that could be "played back" inside the OASIS to perfectly simulate anything a person could experience through their senses of touch, taste, sight, smell, balance, temperature, vibration--you name it. -Ernest Cline, Ready Player Two, Pages 15-16

Being able to give people with disabilities the ability to participate in VR would be revolutionary, as the industry today is currently struggling with creating inclusive designs. Using full dive and BCIs there would be little to nothing preventing anyone from experiencing the technology. Despite these positive applications, it is still important to think about the negative side effects of this technology. BCIs come in direct contact with a user’s brain, giving the device the ability to not only gather all sorts of information about users but also the potential ability to completely take over control of their minds is something that is frightening to most.

The closest thing we have to this technology as of yet is the NextMind BCI. While this device is mainly for developers at this time, it does come with some sample games, several for the computer, and one for VR. The game demos show off the capabilities of the NextMind: A user can move simple components of these games with their mind. One example illustrates that you can control the paddle in a game of Pong, and another shows that you can move obstacles in all directions away from your character. These simple applications are nowhere near as advanced as what the media predicts, but they are a stepping stone to the future of what this technology could provide.

My vision went black for a moment as the headset instructed my brain to place my body into a harmless sleeplike state, while my conscious mind remained active inside what was basically a computer-controlled lucid dream. Then the OASIS slowly materialized into existence all around me, and I found myself standing back inside Anorak's study, where I’d last logged out. Everything looked the same as before, but it felt completely different. I was actually here, physically inside the OASIS. It no longer felt like I was using an avatar. Now I felt like I was my avatar. There was no visor on my face, none of the faint numbness and constriction you always felt wearing a haptic suit or gloves. I didn't even feel the ONI headset my real body was actually wearing. When I reached up to scratch my head, the device wasn't there. -Ernest Cline, Ready Player Two, Page 12

Another example of a BCI technology in current development is Neuralink. Neuralink is a neurotechnology company that was founded by Elon Musk in 2016. Their main goal is to create an implantable BCI that will help people with paralysis. They hope to one day be able to create a full dive VR system to better the lives of people living with disabilities.

In February 2021, Musk released a recording of a monkey that could play video games with its mind by using a Neuralink computer chip in its skull. Musk claims that one day the Neuralink could allow humans to send concepts to one another using telepathy, or even allow people to exist in what he calls a “save state” meaning that after they die, their consciousness could be transported to a robot or another human.

While the concept of full dive is a lot scarier than that of haptic immersion, its applications are endless. One potential use case could be for those who are physically disabled, such as the producers of Neuralink suggest. Most VR experiences at the present day are not very accessible to those who do not have control over their body’s full range of motion. With full dive, however, there is no need for physical movement at all, making this potentially the most accessible type of technology. Another ethical concern for full dive VR is regarding the data it collects, which is a major issue for all the technology we use. This is especially concerning for full dive VR due to its direct connection to a user’s neuro activity.

Conclusion

Although full dive has a lot more controversy surrounding it due to its depiction in mass media, both full dive and haptic immersion have their benefits and drawbacks.

Haptic immersion allows for a more realistic physical experience while also allowing the user to be immersed in other worlds. The user can run, jump, walk, and feel every sensation, every punch, every tap on the shoulder. Full dive connects directly with the user’s mind, making for a more out-of-this-world experience. The user doesn’t need to move a single muscle to experience the simulation, and depending on the programming, they could enable or disable various sensations.

Haptic immersion allows for applications in physical therapy, strength training, and virtual sports, while full dive would allow disabled users to experience VR without the limitations they face in the real world.

Haptic immersion is most likely the first of the two technologies we will see become mainstream due to the wide variety of resources that are already available and will likely remain the one that is most readily accessible and accepted by society. Full dive will provide a much more unique experience for users but will ultimately be the one that undergoes the most criticism and

Personal Take

I am personally very excited to see where these two technologies go as we progress even further in innovation. The two paths are unique in their own way, and I look forward to watching as the paths diverge even further. Of the two, I am more interested in the haptics side of things, due to all of the negative media surrounding the concept of full dive. I’m not quite ready to give up my consciousness to a machine, but maybe by the time we get there, my opinion will have changed.

At the end of the day, knowing the difference between these two types of VR is important as the technology progresses into the future.

About this Author

Ayiana Crabtree, Karp Library Fellow, XR Researcher
February 2021 - May 2022
Area of Focus: Conducting research to find ways humanities disciplines can benefit from XR technologies and developing future programming for Studio X that will engage humanities faculty and students