Real emotions in virtual worlds: Using Virtual Reality to explore the human mind

Why would we do such a thing to people? We are emotion researchers at te Herenga Waka Victoria University of Wellington. For the past five years, we have been using immersive virtual reality (VR) to simulate real-world situations so that we can better understand what emotions are and how they work. Until now, most emotion research has been conducted in sterile laboratory environments where research participants look at pictures or listen to music or watch film clips to evoke emotional states. They may also be held immobile; wired up to record their heart rate and sweaty palms, or to record their brain activity with EEG. Although we’ve learned a fair bit about emotion this way, it’s a far cry from the real world, where our emotions are complex states that guide our perception, memory, and decision-making, prepare our bodies for action, and give rise to a wide range of feelings.

Virtual reality has been a game-changer for emotion researchers. Participants in our plank simulation experience authentic fear – as indicated by their verbal reports, but also their heart rate, skin conductance (sweat), cautious movements, and neural responses. VR technology automatically records the user’s head, eye, and body movements, and new wearable technologies make it possible to simultaneously record brain activity and physiological signals. These continuous measures allow us to capture emotional responses as they unfold over time. Virtual environments are also easy to control – we can easily change the height of the plank or add tasks for people to perform. Our paradigm can be used to study basic tenets of fear, but also to understand how fear affects our thought processes.

Snapshot VR experiment from right (photo supplied)

One consistent finding is that people perform surprisingly well on many cognitive tasks even when they are clearly afraid. Our findings challenge contemporary theories that suggest that fear impairs the ability to think. But we do find that the inverse is true – fear may not affect cognition, but engaging in cognitive tasks does reduce fear. When we give them a challenging task, people show a clear reduction in their fear response, highlighting the value of distraction as an effective way to regulate emotions.

We don’t just scare people in our research. We induce other emotions as well: In one of our simulations, participants navigate through a dirty bathroom (complete with cockroaches, a wet carpet, and an overflowing toilet) to help us understand disgust. And in another set of simulations, we inspire feelings of awe by putting people in vast and majestic environments, like a mountain peak, the ocean depths, or a star-filled galaxy, to determine how awe affects our response to stress. We have worked with Ed Davis, an award-winning VR film-maker and developer, to create compelling virtual worlds that induce a range of powerful emotions. In a new spin-off project, we are using this approach to study how awe-inspiring experiences of Aotearoa New Zealand’s landscapes might promote pro-environmental attitudes and intentions.

Beyond our scientific outcomes, our project has provided research training for a large team, with outcomes including a PhD thesis, four MSc theses, and hands-on experience for more than a dozen honours and undergraduate research students. Our interdisciplinary team includes students and research assistants in Psychology, Cognitive and Behavioural Neuroscience, and Engineering, and they have learned skills that can be applied in both research roles and in Aotearoa’s growing VR industry. The project has also allowed us to build a state-of-the-art VR research lab that can now be used to address dozens of research questions in the psychological sciences. We are just beginning to tap the potential for VR to yield new insights about human minds, brains, and behaviours.

Gina Grimshaw is the PI and David Carmel an AI on the Marsden-funded project “Attention in Emotional States: Insights from Virtual Reality”. Christopher Maymon was the named post-doc on the project and is currently a Lecturer in Psychological Science at te Herenga Waka Victoria University of Wellington. Chris and Gina are co-editors of the book “Virtual Reality in Behavioral Neuroscience: New Methods and Insights”.