Street View-style Imagery Lets Mobile Devices Locate Themselves Indoors As Accurately as GPS

As reported by MIT Technology Review: Smartphones locate themselves outdoors using a GPS sensor, but those signals are typically blocked indoors. A new technique uses a device’s camera to get an indoor location fix to an accuracy of within a meter. The technique could enable new kinds of apps, and may be particularly valuable for wearable computers such as Google Glass.

The new location-fixing method is being developed at the University California, Berkeley. It uses a photo from a device’s camera to work out the location and orientation of the device. It does this by matching the photo against a database of panoramic imagery of a building’s interior, similar to the outside views offered by Google’s Street View. The system can deduce the device’s location because it knows the position of every image in that database.

The researchers used a special backpack that captures Street View-style imagery indoors as the wearer carries it around. It has two fisheye cameras, laser scanners, and other sensors. Software uses the data collected to generate a map of the building’s interior, a stitched-together set of panoramas, and a database of individual images that can be used for location lookups.

“You can provide that blue dot you see on a mobile map when out-of-doors for interiors,” says Avideh Zakhor, who leads the Berkeley group developing the system. Zakhor previously sold a 3-D city mapping company to Google that became a major part of the company’s Google Earth 3-D virtual globe.

Zakhor and colleagues have tested their system in buildings on the Berkeley campus and in a mall in Fremont, California. In tests at the mall, they successfully matched more than 96 percent of images taken by a smartphone’s camera against the database of images. When the matches were turned into location fixes, most came out with an error of less than a meter from the device’s true location.

Zakhor says her approach compares favorably with competing methods of determining location indoors in terms of accuracy and the cost of deployment. Alternative methods include using Bluetooth “beacons” or fingerprinting the pattern of Wi-Fi signals inside a building.

Jonathan Ventura, senior researcher at Graz University of Technology, Austria, agrees. “The major advantage of image-based localization is that it works almost everywhere and doesn't require changing the environment in any way,” he says.

Zakhor’s group isn’t the only one capturing such data: Google has begun taking its Street View product inside and announced last month that it had documented the interiors of 16 airports and over 50 train stations.

Ventura’s own research focuses on augmented reality. He says that if devices can be located very accurately it will allow for virtual and real worlds to be closely aligned. “If we want to render a rich and complex virtual world into a high-resolution image,” he says, “we need to have much more accurate positioning than a consumer GPS receiver can deliver.”

Zakhor is planning tests of her method on computerized glasses, with the intention of having the devices use snapshots to track their location, making it possible to provide a map of an interior space in a person’s field of vision. The Berkeley research group is also working on using data from Wi-Fi signals collected by their backpack to provide a secondary method of deducing a device’s indoor location.