Magic Symbol and Video Conferencing: Expert Help Over Distance
Magic symbol codes, sometimes called quicktime codes, are little squares with a grid on the surface, filled with smaller squares of black or white, arranged in a unique pattern. They serve as unique markers for machine vision systems. Usually as placeholders to tell augmented reality systems where to sight specific virtual objects so they blend in with the physical.
Still, magic squares are a powerful technology for AR, and they don't have to be used simply as placeholders. An alternate use is as labels. Codes telling the camera exactly what the physical object they are looking at actually is, without requiring the processing power in the system to take and study the object's actual three dimensional shape. Perfecrt if you are looking for a minimum processing-power, minimum bandwidth system such as is the case in videoconferencing out in the field.
In this case, the system is being used for expert help with machinery breakdowns. Such breakdowns can occur anywhere. From the assembly line to the back of a farmer's field in the middle of nowhere. The person or persons on-site can attempt to fix it, but all-too-often what is needed is a phone call to the manufacturer, and a long and often unpleasant conversation on the phone as the persons on-site attempts to explain the physical problem in words, to the often-confused person on the other end. How much easier it would be if the person on the other end could see the equipment running and see the problem for themselves.
Developed by the Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE in Wachtberg, this first-generation, crude system attempts to do just that.
A laptop possesses a camera and screen setup mounted on a swivel arm so that the technicians can view the screen while carrying out repairs. An image processing program calculates the cameras position and direction of view and sends this information to the manufacturer over standard telecommunications networks.
This enables the experts to view the machine on their monitor from the same perspective as the technicians. They can even use the software to write instructions on specific parts of the machine such as Remove this screw. These instructions then pop up on the technicians screen on exactly the same part of the machine. With the aid of magic symbol codes carried on the person of the on-site technician, screwed down to the chassis of the equipment permanently, or printed out then and there, and stuck on, the system is able to compare the outline of the equipment to the codes, and create an internal 3D model.
With code cards at all angles around the equipment, wherever the camera moves, at least one is still visible. This means that if the technician walks around the machine with the laptop, the image moves accordingly and the written information stays where it was intended to be, for example hovering over a specific screw, whose placement matches the placement of the 3D model generated by the placement of the magic symbol codes.
Once the technicians have carried out the experts instructions, the pop-up information can be deleted by simply clicking on it. The system is based on a chat protocol, which means everyone involved can communicate either through the chat function or by telephone.
The researchers managed to minimize the quantity of data transmitted to allow the system to function over a cellphone network. That means there is no need for a broadband connection, so technicians can call on experts even from remote locations such as wind turbines in the middle of a field or machines in newly industrialized and developing countries. We only transmit location data, not pictures, says Dr. Thomas Alexander, who heads up the research team at the FKIE.
The researchers have already developed a prototype of the system, and the next step is to carry out a study in which users will put the system through its paces. The results should help the scientists to optimize the system and tailor it more closely to user requirements.