A study led by Dr Manos Tsakiris at the Department of Psychology, Royal Holloway, University of London, and released this month, raises further questions about the nature of self, as it approaches the rubber body illusion issue from a fairly unique angle, providing further evidence that who we are and who we perceive ourselves to be, are not as intertwined as once thought, and both are subject to change based on sensory stimuli.

The argument basically opens up the possibility of a rubber face illusion, to the list of such conditions that can be created successfully when a virtual surrogate and physical sensation are paired. Or in other language, when a volunteer looks at a 'mirror' that is in reality an augmented reality display screen, displaying a different face to the user's own over the area where their face should be which morphs into the same expressions as the user's own face, how does this affect the user's sense of self? Do they come to believe that the face they see in the 'mirror' is really them, and does their self-perception alter because of this?

Traditionally, the perception that the body you see and feel is you, has been estranged from the perception that the face you see in the mirror, and responds to your muscle movements is you. The two have been treated as entirely separate issues.

This work was intended to focus on the psychological nature of the study of self. Thanks to the pace of technological change, coupled with recent advances in facial overlay technologies, the opportunity has finally presented itself to test what was previously only seen as theory.

Of course, those of us who are familiar with the mind to mind of direct, frequent online interaction and self-exploration, know that neither the birth body nor the birth face is not necessarily who an individual actually is. However, empirical evidence to substantiate such beliefs is always necessary.

The Three Experiments

Three experiments were used in succession to test the user's self-perception using the augmented reality mirror. The first and third test being identical in nature, and used to provide measurable data of the effect of the second experiment, the main deal.

Pre-Tactile Test

The subjects sat opposite an AR display screen - a laptop in this instance. For the first experiment, the subjects were to watch a two minute long gradual morph of a picture of their own face.

The original image being 100% them, and the final morph being an aggregate of 50% them, and 50% another face. This was performed for all subjects, with them under instruction to stop the movie playing the moment they felt that they were no-longer looking at their own face.

The same movie was then replayed in reverse for two minutes, again with the participants clicking to stop when they felt they were looking at their own face. The two sets of results were averaged to find the cut off point for each subject.

The image below shows snapshots of the morphing process at 12 second intervals, for a single subject.

Image credit: PLoS ONE

Tactile Test

The tactile test was actually two different experiments, each of which was undertaken by half the study group. One test was synchronous, the other asynchronous, and the differences compared.

The basic experiment was simple. The subject was to stare straight at an image of their own face on the computer, not looking away. At the same time, a researcher would press a pen lightly, to the subject's cheek.

For the synchronous test, a virtual pen was added to the display image, and moved in in sync with the researcher's pen, via a webcam connected to the computer, such that it would visually overlay on the face to match the researcher's pen movements.

For the asynchronous test, a virtual pen was used which did not move in tandem to the physical pen, but followed its own path around the face, resulting in a disjunction between the subject's visual and tactical perceptions.

Top: Subject stares at the display of a face other than their own, whilst a synchronous virtual pen is drawn on it, that moves in conjunction with a physical pen, lightly stroked on the face outside of visual perophery. Bottom: A close-up view of the process, showing the displayed image. Note that the face on the screen is the same mix of 50% the subject, 50% another face that was used on the earlier morph test. Image credit: PLoS ONE

Image credit: PLoS ONE

Post-Tactile Test

After the tactile experiment was concluded, in both synchronous and asynchronous cases, the first experiment was run through again, for each subject. They watched a two minute long gradual morph of a picture of their own face.

The original image 100% their own face, and slowly morphed over the two minutes into an aggregate of 50% them, and 50% another face. This was the same face from before, shown at the end of the first experiment, and shown during the tactile test. As before, the subjects were under instruction to stop the movie playing the moment they felt that they were no-longer looking at their own face.

The same movie was then replayed in reverse for two minutes, again with the participants clicking to stop when they felt they were looking at their own face. The two sets of results were averaged to find the cut off point for each subject.

In this graph:
Black represents the portion of the video felt to be another's face
White represents belief that the morph was still the subject's own face
Red represents the change between experiments one and three

Image credit: PLoS ONE

When the two sets of results from test 1 and test 3 were compared, a marked difference was noticed. In all cases where the visual movement of the pen did not match the feeling of it along the cheek, there was next to no difference in cut off point between the original and morphed face where the subject felt they were no-longer looking at themselves, but at a stranger.

However, when the pen moved in perfect sync both visually and hapticly, a pronounced tendency to bond with the new face as a self-image was evidenced. Similar in function to the rubber hand and rubber body experiments of the past, the combination of touch and sight lent a feeling of the new face being the person, to the proceedings. That difference, is highlighted in red in the graph above.

Concluding Thoughts

This was at heart, a rubber hand illusion study, concentrating on the face. That did make it unique, as the face is one of the most expressive parts of the human body, with the most subtle inflections, and most frequently examined in social interaction and self-grooming. However, fundamentally the experiment is no different from the others that have transpired in that regard. The most recent of which, was only published a month prior.

However, this approach was unique in that it tackled the "self image in the mirror" angle, that most people deal with on a daily basis. It also demonstrated in as little as ten minutes per person that even such daily exposure to self-facial-image can be overridden with relative ease once multisensory stimulation is involved.

The discovery of such a high proportional response to a single session, in which subjects began to bond with the new face to such an extent after a single session, correlated with other rubber hand and rubber body illusion studies. However, what would make for a lovely extension, would be a study on repeated exposure. Would repeat sessions of experiments 2 and 3 over time, create an increasing empathy and familiarisation with the modified avatar as being a part of the true self-image, or would improvements tail off over time?

The suspicion of this researcher is that the improvements would continue to grow until the modified face was perceived as an avatar of the person as much as the physical face is - after all, that is what transpires in social VR, all the time. However, no solid empirical evidence has been gathered to support this theory, and no such experiment is being planned at this time.

Overall, this is however another piece to the puzzle, proving yet again, that no matter which part of a person's body you replace with a duplicate, so long as multiple sensory channels are in place, one of which is visual, and a high degree of correlation to 'human' is present, then a subject's brain will subconsciously react to see whatever changed form is in front of it, as itself.

Good news indeed for avatar bonding, and living as one's avatar.

References

Looking for Myself: Current Multisensory Input Alters Self-Face Recognition - PLoS One original paper

Changing Self Perspective with VR

'It takes 2 to know 1': Shared experiences change self-recognition

Book Quote: Count Zero by William Gibson, Page 256