Stroboscopic Memory Training
Researchers at Duke University have uncovered a system of visual memory training that is easily within the capabilities of modern VR systems. They stumbled upon it, whilst effectively using shutter glasses to view the physical world.
Participants in the study, which was led by Greg Appelbaum, assistant professor of psychiatry at the university, engaged in physical activities, such as playing catch, while using one of the following two optical choices:
Participants completed a memory test that required them to note the identity of eight letters of the alphabet that were briefly displayed on a computer screen. After a variable delay, participants were asked to recall one of the eight letters. On easy-level trials, the recall prompt came immediately after the letters disappeared, but on more difficult trials, the prompt came as late as 2.5 seconds following the display. Because participants did not know which letter they would be asked to recall, they had to retain all of the items in memory.
Humans have a memory buffer in their brain that keeps information alive for a certain short-lived period, said Professor Appelbaum.Wearing the strobe eyewear during the physical training seemed to boost the ability to retain information in this buffer.
The strobe eyewear disrupts vision by only allowing the user to see glimpses of the world. The user must adjust their visual processing in order to perform normally, and this adjustment produces a lingering benefit; once participants removed the strobe eyewear, there was an observed boost in their visual memory retention, which was found to last 24 hours.
Earlier work by Appelbaum and the projects senior researcher, Stephen Mitroff, had shown that stroboscopic training improves visual perception, including the ability to detect subtle motion cues and the processing of briefly presented visual information. Yet the earlier study had not determined how long the benefits might last.
Since all virtual reality systems are stroboscopic by their very nature, this is one memory training exercise that would transfer exceedingly well to any VR environment or installation. No VR system is capable of showing a continuous view of the world. Rather, you get periodic updates which correlate precisely to the frame rate in that environment. Normally, the objective is to get the frame rate as high as possible certainly above 60-70 frames per second so that it updates more swiftly than the human eye can track, resulting in the illusion of a continuous view to the human visual system.
When the frame rate drops, everything appears disjointed; like a series of still images in sequence. It is precisely this effect the experiment has replicated. So, if you slow the rendering rate (which has a nice side-effect that you can render much more detailed environments) then you are once again in the situation where the user's vision is disrupted. Only, because it is taking place in the software of VR, rather than a fixed pair of glasses over your head, the delay is configurable in real-time. It also opens up the accessibility of the experiment to anyone with a basically competent VR system which includes many home gaming PCs. It also opens up the albeit distant possibility of using the same memory training trick on those whose visual system does not function properly, as it has long been known that it is possible to use a VR system to supplant a user's natural senses.
Dangers the study did not consider
The Stroboscopic Effect
Stroboscopic systems have been studied for decades by VR researchers, precisely because they are such a fundamental part of how our display systems work. As such, there are potential dangers to be aware of, which a virtual system can lessen if not completely eliminate, but which could pose a problem in the physical world.
Firstly, there is the age-old issue of the stroboscopic effect, also known as temporal aliasing. It is a nasty bit of visual trickery, which confuses the senses.
It is a kind of aliasing that occurs when what should be a continuous motion such as a wheel spinning, is instead divided into a series of frames per second, which are then sent to the eye. If the movement speed of the wheel is accidentally synced to the update frequency of the frames, then it appears to not be moving, even as the vehicle drives forwards.
In VR this is immersion-breaking. In the physical world, it is potentially hazardous, as your eyes are telling you that vehicle is stationary, and you won't get another update for half a second or more. Plenty of time for things to go disastrously wrong. Common sense should tell the average user not to use a stroboscopic system near roads or other areas with moving vehicular traffic. Unfortunately, as most of us are aware, common sense is not as common as the name would suggest.
Moving the interface into a virtual environment would all-but eliminate this problem, as with current technology, carrying a virtual environment with you as you walk down the road is not an easy task – and if you manage it, you have far more things to worry about than not seeing properly. Not seeing or hearing the outside world at all being a big one.
A second danger is that of simulation sickness. In VR work, this too is an old 'friend'. Simulation sickness occurs when there is lag in the system. Even slight but perceptible lag will do it. If for example, the visual data the eyes are giving the brain, does not agree with the inner ear, in that the eyes say the person is standing still as the frame has not updated yet but the inner ear says they are moving, then this creates an internal sense conflict.
The common response to this is a form of motion sickness or sea sickness in which the user develops dizziness, loss of balance, and in many cases is literally sick. A stroboscopic system such as this one is inherently very susceptible to simulation sickness, and were you to use it for 20 minutes or more, you would feel it, in more ways than one.
Again, VR systems have a partial solution to this, but one in which the home PC is no help. A medium to high-end modern VR installation can sync the input to the inner ear in time with the visual input. Since there are no sensory conflicts, simulation sickness does not occur, no matter how long the session lasts.
The limiting factor here is of course, the limited availability of galvanic vestibular stimulation interface devices the interfaces that can link the inner ear to the simulation. At present they are out of the possibility range for the home user, but does allow for the possibility of this type of training at any one of the growing number of high-end VR facilities.
Terminology (local links)
Stroboscopic visual training improves information encoding in short-term memory (Subscription required)