Google Lookout App to Help the Visually Challenged

With a view to help the visually challenged, Google is developing a new Android app called Lookout, the company announced on the sidelines of Google I/O 2018. The app, which is heading to the US later this year, will give visually challenged auditory cues as they encounter objects, text, and people around them. In March, Microsoft took a similar move and brought Soundscape as its app to help visually challenged navigate cities using audio cues and labels.

The Lookout app has four distinct modes, including Home, Work & Play, Scan, and Experimental. Depending on the mode, the app processes items of importance in the environment and then shares information it considers to be relevant. Google says that the information could be a recipe book, or the location of a bathroom, an exit sign, a chair, or a person nearby. Additionally, the app offers spoken notifications that are touted to be designed for minimal interaction and let users stay engaged with their existing tasks and activities.

If a user selects the Home mode, the Lookout app will provide notifications about the location of the couch, table or dishwasher. Similarly, the Work & Play mode warns users when they're next to an elevator or stairwell, while the Scan mode is designed to read text from a notebook. There is also the Experimental model that allows users to get an early access to the features that are in the pipeline.

Instead of navigating in a traditional manner, the app gives an idea of where the objects are located in an easy way. For instance, it offers "couch 3 o'clock" to let users understand that the couch is on their right.

Users are recommended to use Lookout with their device worn in a shirt pocket or hanging on a lanyard around their neck, with the camera sensor pointing away from the body. This enables the app to scan the surroundings. The app also uses machine learning to learn what users want to hear about and then accordingly deliver more appropriate results. Moreover, the core experience on Lookout is processed on the device to offer its features without requiring an Internet connection. "Accessibility will be an ongoing priority for us, and Lookout is one step in helping blind or visually impaired people gain more independence by understanding their physical surroundings," said Patrick Clary, Product Manager for Google's Central Accessibility Team, in a blog post.

Similar to Lookout, Microsoft's Soundscape also offers a bunch of features for visually challenged people. The app has the abilities to call out key points of interest (including roads and intersections), place an audio beacon on a point of interest, describe the current location, and detail nearby points. However, Soundscape is so far exclusive to iOS devices, whereas Google's Lookout will be available for the Android ecosystem.

We discussed Android P, Google Assistant, Google Photos, and also the most important things that Google did not mention during its I/O 2018 keynote, on Orbital, our weekly technology podcast, which you can subscribe to via Apple Podcasts or RSS, download the episode, or just hit the play button below.