The Minds Eye, Scanning, and Improving VR
If you are looking for someone in a crowded scene, whether a "where's Wally" book, or a crowded cafeteria, your eyes scan the room like a roving spotlight, moving from face to face? Researchers at Researchers at MIT's Picower Institute for Learning and Memory have found that you do. What's more there's something very much akin to a clock cycle controlling the speed at which you do so.
The researchers say that most people scan the room, only a relatively few try to see the full picture all at once. Even they then proceed to scan sections of the room, jumping from face to face, or recognisable object to recognisable object. In other words, during the scan, everything in the room is taken in at a lower resolution, then their positions stored, and the eye's focus jumps from one to the next like a searchlight, narrowing down if that is quite what it is looking for.
In the study, monkeys were given the task of searching for one particular tilted, coloured bar among a field of bars on a computer screen. By monitoring the activity of neurons in three of the animals' brain regions, researchers found that the monkeys spontaneously shifted their attention in a sequence, like a moving spotlight that jumped from location to location.
In addition, the timing of these jumps appears to be determined by waves of activity in the brain that act as a clock. This clock provides a framework for shifting attention from one location to the next in regular, and predictable beats.
"For many years, neuroscientists have been debating competing theories on whether humans and animals spontaneously search elements of a visual scene in a serial or parallel manner," said lead author Earl K. Miller, the Picower Professor of Neuroscience. "Ours is the first study based on direct evidence of neurophysiological activity."
Picower Institute postdoctoral associate and co-author Timothy J. Buschman found that the spotlight of the mind's eye shifted focus at 25 times a second and that this process of switching was regulated by brain waves.
"Attention regulates the flood of sensory information pouring into the brain into a manageable stream. In particular, a lot of different areas of the brain are involved in vision. If they all competed at once, it would be chaos," Miller said. "Brain waves may provide the clock that tells the brain when to shift its attention from one stimulus to another. Oscillating brain waves may provide a way for several regions across the brain to be on the same page at the same time - very similar to the way computers use an internal clock to synchronise the many different components inside."
The researchers' next step is to expand their search for brain wave function beyond the visual. They hope to discover whether brain waves are specific to visual function or act as a "general clock" for the brain.
Regardless, this work opens up a new possibility for VR. If we use an interface than monitors the focus of the eye, and its movement - interfaces which we have had for over a decade - and we are capable of reading that individual's brain waves, which again, we are, then it becomes possible to sync up to their internal clock, and change data accordingly.
For example, if the individual is scanning the virtual environment as they move through it, only the area being scanned would need to be displayed at full resolution - and since scanning typically moves side to side before moving up and down, only a band of the VR would need to be rendered at full capacity. The system can then wait until the next pulse in brainwaves is detected, and shift then to displaying the next section in high resolution, letting the previous fade.
In this way, precious clock cycles could be saved for other uses, whilst the user would never notice the difference.
We are of course, still years from such practical applications.
The study, appears in the Aug. 13, 2009 issue of the journal Neuron. This work is supported by the National Science Foundation and the National Institute of Neurological Disorders and Stroke.