Reverse Engineering The Ability to See
There are a few, albeit exceedingly rare cases, where after a lifetime of blindness, a human's vision is naturally restored. These cases, properly studied, are yielding impressive amounts of data on how a vision system forms in a mature mind - and thus how to recreate the same, in machine vision.
After being deprived of visual input, the brain needs to learn to make sense of the new flood of visual information. A study by MIT neuroscientists suggests that dynamic information - that is, input from moving objects - is critical.
MIT brain and cognitive sciences professor Pawan Sinha, through his humanitarian foundation, Project Prakash (Sanskrit for "light"), has treated and studied several such patients over the past five years. The Prakash effort serves the dual purpose of providing sight to blind children and, in the process, tackling several foundational issues in neuroscience.
The findings from Sinha's team, provide clues about how the brain learns to put together the visual world. Testing the patients within weeks of sight restoration, Sinha and his colleagues found that subjects had very limited ability to distinguish an object from its background, identify overlapping objects, or even piece together the different parts of an object. Eventually, however, they improved in this "visual integration" task, discovering whole objects and segregating them from their backgrounds.
This process is the subject of intense study. Whilst we do not yet know the process how it is done, motion parallax is absolutely key.
One of their subjects, known as S.K., suffered from a rare condition called secondary congenital aphakia (a lack of lenses in the eye) and was treated with corrective optics in 2004, at the age of 29. After treatment, S.K. participated in a series of tests asking him to identify simple shapes and objects.
S.K. could identify some shapes (triangles, squares, etc.) when they were side-by-side, but not when they overlapped. His brain was unable to distinguish the outlines of a whole shape; instead, he believed that each fragment of a shape was its own whole. For S.K. and other patients like him, "it seems like the world has been broken into many different pieces," says Sinha.
However, if a square or triangle was put into motion, S.K. (and the other two patients) could much more easily identify it. (With motion, their success rates improved from close to zero to around 75 percent.) Furthermore, motility of objects greatly influenced the patients' ability to recognize them in images.
During follow-up tests that continued for 18 months after treatment, the patients' performance with stationary objects gradually improved to almost normal.
The team believes this came about via a bootstrap system. Learning of rules and heuristics by which the brain comes to be able to parse static images is triggered by watching the moving objects, and treating every part which moves in sync with the others as part of the same object.
Starting from an initial capability of grouping via motion, the brain begins to notice that similar dynamics are correlated with similarity in other region attributes such as orientation and colour. These attributes can then be used even in the absence of motion.
"If we could understand how the brain learns to see, we can better understand how to train a computer to do it. "