Microsoft's Seeing AI App for iOS Now Lets Visually Challenged Users Explore Photos by Touch

Microsoft has updated its Seeing AI computer vision app for the visually challenged people with an option to explore photos by touching them.

The new feature in the Seeing AI iOS app enables users to tap their finger to an image on a touch-screen to hear a description of objects within an image and the spatial relationship between them.

"Users can explore photos of their surroundings taken on the 'Scene' channel, family photos stored in their photo browser, and even images shared on social media by summoning the options menu while in other apps," Saqib Shaikh, Software Engineering Manager and Project Lead for 'Seeing AI', said in a blog post late Tuesday.

Seeing AI is already helping users read printed text in books, restaurant menus, street signs and handwritten notes, as well as identify banknotes and products via their barcode.

Leveraging on-device facial-recognition technology, the app can even describe the physical appearance of people and predict their mood.

"For the first time we're releasing iPad support, to provide a better 'Seeing AI' experience that accounts for the larger display requirements," Shaikh added.

Since the app's launch in 2017, Seeing AI has leveraged AI technology and inclusive design to help people with more than 10 million tasks.

Here is what's new, as per the App Store changelog:

• Explore photos by touch: This new feature allows you to explore how objects in a photo are arranged. Select "Explore Photo" from the Scene channel, photo browser, or when recognizing photos from other apps. Then, move your finger over the screen to hear where objects are located.
• All new iPad support.
• Get faster access to your favorite features by customizing the order in which channels are shown.
• On the Person channel, you can now teach Seeing AI to recognize someone new, directly from the main screen.
• When recognizing photos from other apps, you will now hear the processing sound as in the main app.