BlindAid: Mirror World for the Blind
Mirror worlds are virtual environments, which as precisely as possible, are complete copies of our physical world. Augmented reality layers virtual data over the physical. BlindAid is a scheme to merge the two technologies, overlaying a perfect recreation of the physical world, over the physical.
Why, you ask? Well, so the computer system knows where every wall, every floor, every ceiling, every manhole cover is, at least in theory. This is then used to create an all-encompassing augmented reality environment, that gives instruction to the blind and partially sighted, as to when to move, and when to watch out.
It is an incredibly ambitious project, the brain child of Dr. Orly Lahav of Tel Aviv University's School of Education and Porter School for Environmental Studies. Still, it is the very embodiment of the kind of help for those with disabilities, that AR and mirror worlds could offer.
On the hardware side, the current interface is primitive, but then what VR interface isn't, at this time? The interface currently consists of a joystick, connected to, a 3-D haptic device, that enables movement through the mirror world, and feedback as to obstacles encountered. The joystick can be replaced with leg or shoe sensors on the individuals concerned of course, but the system cannot account for items placed in the way, and its not that good yet. What it can do, and is being used for, is to familiarise a user with the basic layout of an area before they step into it for the first time.
Not quite as ideal, but not quite as challenging either. It allows an individual with sight impairment to feel places where the floor changes height, where the walls and corridors are, major thoroughfares and so on, so as to build a map in their mind of how many steps down the hall a doorway is, or how wide the pavement is, before they actually visit an area.
The 3D haptic feedback device produces force feedback and a vibrating haptic sense of touch when a wall is encountered, or the floor drops away, or the user has just walked into the opening range of a door.
To aid learning, the mirror world is entirely haptic hased: it has no visual component. This is fine for the truly blind. Gor everyone else, it allows them to memorise the core information without changes to the actual physical environment on a given day, confusing things.
"This tool lets the blind 'touch' and 'hear' virtual objects and deepens their sense of space, distance and perspective," says Dr. Lahav. "They can 'feel' intersections, buildings, paths, and obstacles with the joystick, and even navigate inside a shopping mall or a museum like the Louvre in a virtual environment before they go out to explore on their own."
Mapping the physical locations is as simple as utilising existing GIS (Geographical Information Systems) technologies to map a VR recreation out. So far the university campus and surrounding streets have been mapped out to a level of detail that allows non-university users to explore the physical campus wearing a thick blindfold, not cheating, and still not getting too lost.
The tool transmits textures to the fingers and can distinguish among surfaces like tiled floors, carpet, asphalt, concrete and grass using crude changes in vibration for the different textures, allowing the textures a user's feet usually feels, to become landmarks and placeholders. The mind learns for example, that when in a given corridor, when the surface changes to carpet after lino, there is a staircase straight ahead, 50 footsteps away.
The virtual system becomes a computerised "white cane" for the blind, says Dr. Lahav. "They get feedback from the device that lets them build a cognitive map, which they later apply in the real world. It's like a high-tech walking cane," she says. "Our tool lets people 'see' their environment in advance so they can walk in it for real at a later time."