Use virtual reality for a change

In this article, I will write about using VR for a change in two different ways:

Use Virtual Reality for a change

Use Virtual Reality, for a change.

Take a look at this ABC News Report from 1991:

There are several things to note about this. First, it is familiar – it reflects very much what is being said today, that virtual reality is going to “change the world”, that it is a new technology with a huge promise, that there is massive excitement about it. Second, people today think that VR back in 1991 was really bad. Truth is it wasn’t and still isn’t. It was, though, really very expensive and highly inaccessible to the mass of people. It was confined to a few well-off research labs around the world, and also had a place in industry.

For example, in 1991, around the time that I started research in this area, in the UK there were maybe 5 Universities at most that had this capability, and maybe roughly the same number in the rest of Europe. You needed funding of hundreds of thousands of pounds/dollars in order to carry out research in this area. The head-mounted display was very expensive, but more than that you needed a very powerful computer to be able to drive the graphics in real-time. This is especially the case because VR is a multi-processing system, taking care of graphics, real-time tracking, interaction, the state of the graphical objects in the environment. You could not just run it on the equivalent of a ‘PC’.

The first computer we had for this was in fact a parallel system specially developed for VR by Division Ltd / Division Inc. which was a UK company specialising in VR Hardware and software at that time. It even had its own specialised operating system. Division developed its own HMDs (Provision).

It became clear during the 1990s that VR was not going to move into the public domain, and that developments in HMDs were not really forthcoming. VR disappeared from public view, and my own research group moved to using a Cave system (installed at UCL around the year 2000), and driven by a very expensive Silicon Graphics machine. This was how we survived carrying on our research during the 2000s, using a Cave-type of system. To give an idea of the cost, the SGI computer system and the Cave itself each cost approximately half a million pounds (made possible by a special UK government infrastructure grant to UCL).

It should also be noted that to install the Cave, a crane had to be used to lift up the components to the Pearson Building at UCL, and parts of the walls of the building had to be knocked out to allow delivery. So, in spite of all its promise in the early 1990s, revolutionising computing, revolutionising the world, it became abundantly clear that VR was something that was not going to move into the home! By the way, someone who contributed greatly to the durability of VR, and who belongs as one its most prominent members of the VR Hall of Fame, is Dr Carolina Cruz Niera who, with others, was responsible for inventing the Cave type of system on which many people relied during the dark days of VR when it was thought to have been a “dead” field.

But why is VR so powerful? What distinguishes it from other forms of media? I believe that it affords three special illusions.

Immersion (physics of the system)	Illusion	Interpretation
Sensorimotor contingencies	Place Illusion	“I am here”.
(i) Responsive (ii) Personal (iii) Congruent	Plausibility	“This is really happening”.
Bodily multisensory integration	Virtual Body ownership (and agency)	“This is my body (and I am doing this action)”

The first column simply describes objective facts about what the VR system delivers. ‘Sensorimotor contingencies’ refers to how we move our body for perception. We move our eyes and head and we see and hear something different or in a new way (e.g., closer, louder). We bend down or turn around and new fields come into our visual and auditory perspective. We reach out and touch something, and we feel it. To the extent that a VR system makes these possibilities available, so it supports natural sensorimotor contingencies. This integrates many aspects of the VR system – visual and auditory display, tracking, and haptics.

In real life, when we look close up to an object, it does not dissolve – but in VR it is likely to dissolve into pixels. In real life, we see approximately a field-of-view of approximately 180 degrees (slightly more), with high acuity at the centre (foveal vision) and low acuity at the periphery. In real life, we see close up objects in stereo. Hence, stereo, resolution and field of view certainly contribute to the type of sensorimotor contingencies that any VR system supports. Now to the extent that the VR does support natural sensorimotor contingencies so emerges the likelihood of ‘Place Illusion’ (PI) which is the illusion of the participant of being in the place depicted by the VR displays: “I am here”.

PI is often called ‘presence’ that ‘sense of being there’ which is the most compelling initial experience that people have of VR: the “wow!” experience. But there is another component, vitally important, that is often overlooked. PI refers to the form by which we perceive the VR. However, what about the content, what we perceive rather than how we perceive it? The extent to which the VR is responsive to the actions of the participant, and there are events that spontaneously refer directly to the participant, is completely determined by the programming of the system.

If in VR the you see a virtual human character, and you move close to it and hold out your hand, absolutely nothing will happen in response to this unless it has been programmed to do so. No virtual human character will ever look at you, or say “Hi” unless this has been programmed. Moreover, VR applications, for example, for training, are designed to simulate potentially real events, events that could happen. Whenever this is the case it is vital that fundamental elements that would be expected by participants must actually be programmed to be there or to happen.

For example, if an application is for the simulation for medical General Practitioners to experience certain types of patient interactions, then every GP these days expects a computer to be on their desk that they can refer to while talking with the patient. If this computer is not there, the application loses credibility. These three elements (at least!) – the environment being responsive to you (a crowd parts as you walk through it), personally related to you (e.g., a character spontaneously looking to you and smiling as you walk by) and satisfying domain expectations (e.g. if it is a football crowd they should be dressed and behave appropriately) – result in the illusion which I call ‘Plausibility’ (Psi). The possibility for Psi is something that again is completely determined by the ‘physics’ of the system – if the hardware allows it and the program supports it. It is interpreted as “These events are really happening” which if you think about it is not really the same as “I am here”. (You can be somewhere, but not believe what is apparently happening – e.g., when you figure out how a magic trick is working, or if you realised that you are being defrauded – led to believe something that is not true).

When you wear a head-tracked and head-mounted display and look down towards your body, what do you see? If it has been so programmed you could see a virtual body, spatially coincident with and substituting your own. If the hardware and software supports this, when something is seen to touch your virtual body, you might feel it on your real body. If you move your real body, then real-time motion capture can result in the virtual body moving synchronously. Under these conditions, you are likely to have the perceptual illusion that the virtual body is your body. You will also have a sense of ‘agency’ which is that you are the author of the actions of the virtual body, which will be true to the extent that the real-time tracking and associated software supports this – and you may even have this illusion even if there is a limited tracking capability.

These three illusions: “I am here”, “This situation is really happening” “This is my body” are the core, and highly interrelated illusions that VR directly delivers.

There are two things to consider here:

• First of all, these are not beliefs, they are illusions. When I am in VR and I experience these illusions, I know for sure that they are not true! But they are like any perceptual illusion – knowing that they are not real does not stop them from happening.

• Second, these illusions have consequences – PI and Psi generally result in you behaving and responding realistically to events and situations that occur in the VR. For example, a virtual character exhibits pain, and you are affected by that, even though you know nothing is happening. You see a deep pit in front of you and your heart races and you step back from it, even though you know 100% that nothing is ‘really’ there.

Although VR apparently ‘died’ sometime in the 1990s, a lot has happened over 25 years.

There has been a massive research effort. Type “virtual reality” as an ‘exact phrase’ in Google scholar and there are more than 600,000 hits between 1990 and 2013. Even taking into account duplications, and citations, this is still a lot. If we restrict this to a phrase in the title only then it is still 17,600. Government agencies such as NSF in the US, EPSRC, and the European Union have funded a lot of research. For example, research groups which I have led have received some millions in funding from the EU, not even including the funding that went to our project partners, starting with the project Presencia in 2002, and the most recently finished collaborative projects being Beaming and VERE. Also, VR has been used in industry throughout this period to, for example, support remote collaboration in vehicle design and for a 1999 review. This sums up the “use VR for a change.” It has been used, especially in applications such as virtual prototyping (e.g., of motor vehicles), training (the most obvious being flight simulators). What’s different now is that it can be used far more widely and far more inexpensively than was previously possible.

But what about “use VR for a change”? One of the most substantial uses of VR, especially in research, has been its deployment in clinical psychology. This follows conventional clinical methods but where VR has been used as an adjunct to standard techniques (exposure therapy, and cognitive behavioural therapy). Results show that it is at least as good as conventional therapy but with economical and logistical advantages. VR is useful here because of presence (PI and Psi). Effective treatments require that the patient experiences sufficient anxiety to make it possible for them to learn to reframe and thereby diminish this.

If someone has a fear of spiders they can be shown photographs of a spider, plastic models, gradually exposing them to the feared spider, until eventually after maybe months of exposure they can be in the same room as a real spider. In VR, it is possible to put them in the same virtual room as a virtual spider, and via presence have a suitable feeling of anxiety in close proximity to something that appears to be a spider (but which is not). Presence with a virtual spider is sufficiently anxiety provoking, but knowledge that there is no spider allows spider phobics to do this. Similarly, with fear of heights – patients with a phobia could not be dragged by horses to a real height, and therefore the exposure has to be very gradual. However, in VR they typically will permit themselves to experience a virtual height while nevertheless having sufficient anxiety for the therapeutic process to be effective.

So, VR has been and will be used more and more as a way of achieving personal change. The logistic and economic advantages of VR are that to achieve exposure may be much more cost-effective for clinicians than conventional methods. If someone has a phobia of speaking in public, it is very difficult and costly to arrange sessions where patients can speak in public. However, it has been shown that people do respond with sufficient anxiety to entirely virtual audiences. For example, even experienced public speakers find difficulty in speaking to this audience notwithstanding the simplicity of the graphics.

The promise, excitement and expectation around VR today is highly reminiscent of the 1990s. Then it did not survive because both the computing and display hardware was too expensive and inaccessible, so that relatively few labs and industries around the world could participate. Today the computational and display hardware is far more accessible and growing in power and accessibility every year. Moreover, mass communications through social media makes all aspects of development highly accessible – what is available to buy, what is under development, software examples, tutorials, applications.

The dream of Dr Timothy Leary at a Panel in the SIGGRAPH conference of 1990 today seems far more like reality than science fiction: “… suddenly the barriers of class and linguistics and education and nationality are gone. The kid in the inner city can slip on the telepresence hardware and talk to young people in China or Russia. And have flirtations with kids in Japan. In other words, to me there is something wonderfully democratic about cyberspace. If it’s virtual you can be anyone, you can be anything this time around. We are getting close to a place where that is feasible.”

But today there is a different danger: What is VR good for? Why go through the trouble and inconvenience of setting up a system and putting on a head-mounted display, when you might be able to have similar enough experiences just by watching TV, or playing a conventional computer game? A potential problem of VR today is content that is specifically designed for this media. VR must be used as a medium in its own right, with its own conventions allowing people to realise experiences that can only be done in VR. VR is not just a display in 3D. In VR, we are not ‘users’, we are participants in an experience. We don’t ‘use’ virtual reality any more than we ‘use’ reality. We don’t use it we are ‘in’ it, we ‘are’ it, our actions and interactions form the very reality that is virtual reality.

Although there are applications when we are simply ‘users’ (e.g., using VR for the purposes of data and scientific visualisation) and there are applications where simulation of reality is the goal (for example, for training) the mass of the population will not be using VR for such purposes. Moreover, these won’t be the most interesting and stunning applications that give VR its raison d’etre. Let’s take the clinical psychology applications as an example: for the vast majority of such applications VR is a convenience – use a virtual height for fear of heights, use a virtual audience for fear of public speaking. While it is logistically and economically easier to use a virtual height rather than a real height, a real height could still be used. In other words, VR is only being used to simulate reality, it is not being used as a highly unique and novel medium in its own right. Of course, such applications are great, and important, but they are not enough to establish VR as method to deliver new experiences that can result in powerful change and which are impossible in reality.

For example, if you were a super hero, apart from enjoyment of the sheer experience of flying, how would this change your behaviour towards helping others? The group of Dr Jeremy Bailenson of Stanford described an experiment where people who experienced flying around (in VR) like a superhero became more prosocial (helping of others) in a later action compared with others who flew around in a simulated vehicle. In our own group, we showed how we could make the act of talking to oneself like talking to another person, where you could to apparently have a conversation with someone famous like

Dr Sigmund Freud, and both be yourself and be Freud responding to yourself.

Also, can the implicit bias of people towards other racial groups be diminished by experiencing having ownership over a virtual body that looks like a member of that racial group? The answer appears to be yes, with several scientific studies all coming up with the same results. You can even experience an out-of-body experience through VR, and this seems to diminish death anxiety.

My overall conclusion is that with respect to system developments, hardware, software, platforms, VR is moving forward at a great pace. However, for it to be taken up in the mass market it has to be more than just transporting computer games from the flat screen to VR and more than just passive observations of events, however fascinating and remarkable.

VR must develop its own paradigms, its own ways of allowing us to experience situations and events that we can never realise in physical reality, and which can result in profound changes where these are changes along the lines of “that was an amazing experience that really makes me feel good”, “more profound than the best movie I have ever seen”, “an amazing experience and I learned a lot from it that I otherwise could have never known”. “I learned about this historical event not simply by observing it, but by being part of it”, “I somehow was the equation that I was trying to understand” and so on. This is using VR for a change.

Finally, VR is such a different medium that many of the usual methods for human-computer interaction break down. It may not be well-remembered that our modern methods for human-computer interaction largely arose out of 10 years dedicated research at Xerox Parc in the 1970s. A mouse, menus, point-and-click interfaces, windows – all the things we take for granted in modern computer interfaces are great for 2D displays. They are almost absurd in VR.

In my next Blog piece, I will write about the challenges of interaction in VR. However, in the meantime think about using virtual reality for a change and you’ll soon see that it is useful for discovering new ways of solving many problems in everyday life.

Use virtual reality for a change – its power resides in being able to change and enhance ourselves through highly novel and physically impossible experiences.

Further reading