360 degree Full Motion Video Panoramas
Virtual tour cameras have existed for a number of years: 360 degree cameras that look in practically every direction at once, and take a snapshot, freezing time into a moment that can be explored at a later date, in a non-interactive series of virtual locations.
However, what these cameras lacked was the ability to take frame after frame after frame of such images, building up a 360 degree video source. Perhaps predictably, whilst that is no-longer the case, the breakthrough did not come from any facet of the VR fields, but from security systems instead.
The merger of VR and crime is nothing really new; the ability to reconstruct a crime scene into a virtual model has been a staple of TV shows for only slightly longer than the real thing. However, a new video surveillance system currently being developed by the US Department of Homeland Security's Science and Technology Directorate promises to go a stage further for both security, and this type of VR.
A traditional security camera is very limited. It can only face in one direction, and if it has to zoom in on any area to increase focus, the rest of the scene is sacrificed. A 360 degree camera solves this issue, but it needs to be a 360 degree camera with more than one lens pointing into any given field of view, so that one can zoom whilst others watch the area.
It needs to use panoramic stitching technology to seamlessly meld the images into one, it has hefty bandwidth requirements that must be managed in as practical a way as possible, it needs to be relatively small, and sturdy enough to survive most attempts at vandalism. It also needs to be fairly cheap to produce. In other words, it needs a great deal.
The new camera prototypes, codenamed ISIS, or Imaging System for Immersive Surveillance attempt to cover this long list of wants and needs. As a by-product, this means they are actually ideal for creating panoramic VRs and other uses as well.
Inside the protective bulb, a series of individual cameras sit, each one pointing into a slightly different direction, each having a fisheye lens, and each one's field of view being overlapped completely by two others. Fisheye lenses distort heavily to either end of the view, but, by having a second camera cover the area of heavy distortion, it is possible to reverse the distortion, relatively cheaply in terms of computational cycles. The view from these cameras is then stitched together into a single panoramic view before it ever leaves the camera.
This cuts down on the bandwidth of the video feed greatly. Rather than having to track each camera independently and piece the image back together at central control - something that would require a great many cables - only one image is continually sent. A single image with a combined resolution of 100 megapixels. More than enough for fine details to be preserved onto the recording.
When a zoomed in view is required, any of the cameras can be 'undocked' from synchronisation, and zoomed in on a target area. However, because the fields of view overlap, no data of the rest of the scene is lost. Additionally, because the image recognition technology is built into the cameras themselves, to stitch the images together, other possibilities have emerged. 'Tag' an object in the view, and a camera will follow it, panning and tilting as necessary, in order to keep the item in focus. If it moves too far out of one camera's view, another takes over seamlessly whilst the original returns to position.
The net result is a full motion video panoramic tour, at anything from one update a second, to 25 frames per second, enough that the scene completely surrounds an investigator and unfolds around them. In other words, the first true virtual reality surveillance system.
With the help of technology experts from the Department of Energy's Pacific Northwest National Laboratory, Lincoln Laboratory has built the current system with commercial off-the-shelf cameras, computers, image processing boards and software.
Obviously these are just prototypes, and the completed system is not yet upon the market. However, the potential applications expand far outside the security field, with possible uses in teleconferencing, art, exploration. Place one such camera on a trolley and move it through a room, and you turn a simple panoramic VR into a panoramic tour. Moving seamlessly from location to location through a room, or across a seabed, allowing those who to view it to feel like they are really there.