Two-way Link Between Sound and Fear Perception
It has long been known that sound can affect our emotions; anyone who has been moved to tears by a strong performance or a wonderful song knows this, as does anyone who finds their spirits lifted by a particular piece of music. The sound becomes associated with an emotion in our memories, so when we hear the sound, the associated mood returns
However, this has a dark side as well. Sounds can trigger negative emotions just as easily as positive ones. Generally found in war veterans returning from service, victims of assaults and the like, certain specific sounds linked to the bad incidents in their past, trigger the same associations and bring the rest of the memories and feelings flooding back.
In war vets, it manifests as harrowing memories of the battlefield, triggered by a sound that could be innocent, but sounds similar to ones from the battlefield. Someone whose PTSD (post-traumatic stress disorder) was a result of being helpless whilst shells exploded all around them, will likely find the sound of thunder to trigger a full recollection of the horrors and fear they felt, since the thunder sounds so much like it belongs on that battlefield.
Whilst there are plenty of other examples of sounds triggering negative memory associations outside of PTSD, the traumatic stress disorder is also very relevant in terms of another discovery about the effects of sounds and associative memory we were previously unaware of.
Researchers from the Perelman School of Medicine at the University of Pennsylvania have discovered that not only is the link between sounds triggering emotions present, but the reverse is also true. Our emotional state at the time, determines how we interpret the sound. The same sound being quite different depending on our initial emotional state.
So, we now have an interesting feedback loop. The person's emotional state determines how they interpret the sounds they hear, and the sound they think they heard, then triggers a new emotional state which then modifies the next sound they hear... and so on and so on.
There are obvious implications here when designing a story-driven virtual environment such as a gameworld, or an educational environment. It is a pathway we were previously unaware of, which can be used to mould the user's emotional state into a feedback loop, by exposing them to a range of sounds at different frequencies early on and creating an association between those sounds and specific events. It then becomes possible to mould the user's experience by using their own emotional states and the immersion of your environment, together with sounds as well as sights, to alter the way they perceive the environment around them. In theory, you could change the very way they are perceiving the world around them by taking advantage of such feedback loops, now we know they exist, and what to look for.
Back in the University of Pennsylvania study, the researchers are looking specifically at the fear emotion response, and how sound feedback triggers fear, and fear then modulates all sounds we hear after that point on. Were you to turn the fear into a different emotional state, those same sounds would trigger very different associations.
"Emotions are closely linked to perception and very often our emotional response really helps us deal with reality," says senior study author Maria N. Geffen, PhD, assistant professor of Otorhinolaryngology: Head and Neck Surgery and Neuroscience at Penn. "For example, a fear response helps you escape potentially dangerous situations and react quickly. But there are also situations where things can go wrong in the way the fear response develops. That's what happens in anxiety and also in PTSD -- the emotional response to the events is generalized to the point where the fear response starts getting developed to a very broad range of stimuli."
Geffen and the first author of the study, Mark Aizenberg, PhD, a postdoctoral researcher in her laboratory, used emotional conditioning in mice to investigate how hearing acuity (the ability to distinguish between tones of different frequencies) can change following a traumatic event, known as emotional learning. In these experiments, which are based on classical (Pavlovian) conditioning, animals learn to distinguish between potentially dangerous and safe sounds -- called "emotional discrimination learning." This type of conditioning tends to result in relatively poor learning, but Aizenberg and Geffen designed a series of learning tasks intended to create progressively greater emotional discrimination in the mice, varying the difficulty of the task. What really interested them was how different levels of emotional discrimination would affect hearing acuity in other words, how emotional responses affect perception and discrimination of sounds. This study established the link between emotions and perception of the world something that has not been understood before.
The researchers found that, as expected, fine emotional learning tasks produced greater learning specificity than tests in which the tones were farther apart in frequency. As Geffen explains, "The animals presented with sounds that were very far apart generalize the fear that they developed to the danger tone over a whole range of frequencies, whereas the animals presented with the two sounds that were very similar exhibited specialization of their emotional response. Following the fine conditioning task, they figured out that it's a very narrow range of pitches that are potentially dangerous."
When pitch discrimination abilities were measured in the animals, the mice with more specific responses displayed much finer auditory acuity than the mice who were frightened by a broader range of frequencies. "There was a relationship between how much their emotional response generalized and how well they could tell different tones apart," says Geffen. "In the animals that specialized their emotional response, pitch discrimination actually became sharper. They could discriminate two tones that they previously could not tell apart."
"We think there's a strong link between mechanisms that control emotional learning, including fear generalization, and the brain mechanisms responsible for PTSD, where generalization of fear is abnormal," Geffen notes. Future research will focus on defining and studying that link.
Their study has been published in Nature Neuroscience, and whilst the majority of potential applications for their work in VR, will not be dealing with PTSD treatments, it is quite clear how valuable the follow-up work will be for us in codifying exactly how the emotional response changes sounds, as well as how sounds change that emotional response. One thing the research to date has also managed to do, is identify exactly where in the brain, both types of sound-emotion interaction are taking place, which will be of value once brain-machine interfaces advance to the point we can use them for far greater immersion environments. For now however, confirmation of the existence of emotionally-triggered modulation of sensory stimuli is invaluable.
We can in fact then use the same sounds to provoke different emotional responses in-world, providing we use the immersion our environments offer, to alter the emotional state of the participant in a controllable way, prior to their encountering the sensory input. Thus we do directly control to an extent, what emotional state the participant will exit the encounter in, based on initial emotional state and prior associations.