Wolfram Alpha's New Website Tries to Identify the Contents of Any Image

Wolfram Alpha might look like a search engine but it's not. Its focus has always been on answering questions rather than displaying links related to search keywords. It can answer questions such as the distance between Earth and Pluto, which is the heaviest metallic element, among many others. Its creator Stephen Wolfram has made a new website that tries to identify the content of any image.

Taking what is called the 'artificial intelligence engine' a step forward, Wolfram Alpha's ImageIdentify service analyses the content of any image, tries to categorise it and sometimes even identifies it. Throw an image of Michael Jackson, and it categorises it as a person, and then identifies him as Michael Jackson. It also provides basic information about the late pop singer such as dates of birth and death.

Stephen Wolfram wrote in a blog post, "'What is this a picture of?' Humans can usually answer such questions instantly, but in the past it's always seemed out of reach for computers to do this." The solution, according to Wolfram, is ImageIdentify. He wrote, "Now I'm excited to be able to say that we've reached a milestone: there's finally a function called ImageIdentify built into the Wolfram Language that lets you ask, 'What is this a picture of?' - and get an answer."

This is not the first time a company has tried to use machine learning to identify content in images. Several photo services have used image recognition technologies to categorise photos into albums. Microsoft's OneDrive and Yahoo's Flickr are two notable examples of photo services using similar technologies to automatically create albums. Facebook has also implemented a similar feature that can identify people in photographs. Amazon Firefly and Google Goggles are two more examples of the big tech companies trying to get image recognition right, hinting that this is a technology that has commercial potential if done right.

(Also see: 10 Apps to Augment Your Reality)

Most attempts to get this done so far have been not been huge successes, so does Wolfram Alpha get it right? Not entirely, as we found out when we used the service. It is good at identifying some people, but not others. It couldn't identify an image of Prime Minister Narendra Modi, but categorised him as a "person", which wasn't very helpful. Similarly, it struggled when we tested it with images of multiple people. An image of Narendra Modi and Jack Ma was classified as "construction" first and then as a "room", both of which aren't true. Another image of Michael Jackson standing with a dog was identified as a "working dog", which kind of misses the point.

We also tried a whole range of images from a BSNL logo (identified as "vertebrate") to Captain America ("instrumentation"), which just proves that the service is a work in progress. Wolfram acknowledged this in his post, "Yes, ImageIdentify could be completely wrong. But somehow the errors seemed very understandable, and in a sense very human. It seemed as if what ImageIdentify was doing was successfully capturing some of the essence of the human process of identifying images."

A service like ImageIdentify relies on usage. The more people use it, the more accurate its results would be - a concept anyone who uses Google search should be familiar with. This also means that the service stores a compressed copy of every image you upload (to improve accuracy). Microsoft's How Old website tries to identify your age based on your photo but it doesn't store the photo. Wolfram Alpha's service isn't like that so you don't want to start uploading your entire personal library of images to ImageIdentify.

The way apps and services use AI-like technologies such as machine learning and behavioural prediction is getting more interesting and powerful all the time. Wolfram Alpha's service could be used by apps that want to help you identify people in photos, sort photos into albums, checking if people are uploading your photos online without permission, among many other things. For now though, you can enjoy the service when it works and have a good laugh when it doesn't.