As reported by MIT Technology Review: Smartphone camera apps could soon do more than just capture images.
Software integrated with a new line of smartphone chips will be capable
of recognizing, say, a cat or city skyline in a photo, and tagging
pictures of your friends with their names.
The
chip maker Qualcomm announced last week that it will bundle the
software with its next major chip for mobile devices. The technology,
which Qualcomm calls Zeroth, could make sophisticated machine learning
more common on mobile devices. As well as processing images, the Zeroth
software is designed to allow phones to recognize speech or other
sounds, and to learn to spot patterns of activity from a device’s
sensors.
The
technology uses an approach to machine learning known as deep learning
that has led to recent advances in speech and object recognition, as
well as software able to play Atari games with superhuman skill (see “Google’s Intelligence Designer”).
Deep learning software is loosely modeled on some features of brains.
It can be trained to recognize certain objects in images by processing
many example photos through a network of artificial “neurons” arranged
into hierarchical layers.
Tim
Leland, a vice president of product management with Qualcomm, says the
company plans to work with partners to build apps to make use of the new
capabilities. He wouldn’t say exactly what those apps might do, but
Qualcomm’s demonstrations of the technology have so far focused mostly
on enhancing the features of camera apps. Last week at the Mobile World
Congress event in Barcelona, where Zeroth was announced, Qualcomm showed
it powering a camera app that could recognize faces it had seen before,
and detect different types of photo scenes.
The
Zeroth software is being developed to launch with Qualcomm’s Snapdragon
820 processor, which will enter production later this year. The chip
and the Zeroth software are also aimed at manufacturers of drones and
robots.
Normally an app has to send data out over the Internet to a powerful server in order to perform such tricks (see “10 Breakthrough Technologies 2013: Deep Learning”).
By doing such computation on the phone itself, the software should be
better at interpreting data from location and motion sensors on a
device, Leland says. He predicts that one of the first applications of
Zeroth will be extending the battery life of devices by tracking the way
a person uses a phone and learning when it could power down to save
energy without affecting the user experience.
Qualcomm
is also experimenting with chips that have physical networks of
“neurons” made from silicon that communicate with spiking electrical
signals (see “Qualcomm to Build Neuro-Inspired Chips”).
That might give devices more powerful learning abilities, but it would
mean they’d work much differently than today’s devices do.