Google Patents Smart Glasses Assistant That Adapts Suggestions Based on User Gaze, Voice Input

Google has been granted a patent for an automated assistant that can be used to deliver suggestions to a user wearing smart glasses and adapt them based on what the user is looking at, or their verbal instructions. While Google was previously working on a pair of smart glasses that relied on augmented reality (AR) technology, the company reportedly abandoned its 'Project Iris' AR smart glasses last year, in favour of building similar hardware for OEM partners. It is currently unclear whether the company will launch a pair of AR glasses based on the technology described in its latest patent.

Google's Assistant for Smart Glasses Could Identify Environmental Features of User Interest

In a document (via 91Mobiles) published on the World Intellectual Property Organization (WIPO) website titled "Adapting assistant suggestions rendered at computerized glasses according to changes in user gaze and/or other user input", the company describes the use of an "automated" assistant that can adapt to audio and video input provided by a pair of smart glasses word by a user.

Google's patent document shows the adaptive assistant in action (tap to expand)
Photo Credit: WIPO/ Google

The company says that the automated assistant would be able to render suggestions on the display of the smart glasses, and that users will be able to select these options with technology that tracks a user's "gaze". This suggests that the device would be equipped with some form of eye tracking that could be used by the assistant.

When a user looks somewhere else, the assistant would be able to use the camera and microphone on the smart glasses to "adapt" its suggestions dynamically, based on the user's point of view and their verbal instructions, respectively.

Google provides an example of the user wearing the glasses, while exploring a foreign city. The device would provide suggestions for restaurants based on the direction of their gaze (Fig. 2B). Users would be able to invoke the assistant by tapping the glasses or using a wake phrase.

According to the document, the assistant would also limit the number of suggestions seen on the smart glasses' display, as too many of the suggestions could obstruct the wearer's view. These suggestions can be selected based on gestures, or using a spoken command. The patent also suggests that the assistant could interface with other applications on the device.

The company also describes the ability to "offload computational tasks" to a server device that would enable the smart glasses to "conserve computational resources" — effectively providing additional battery life. This means that the assistant could be hosted on the server or the smart glasses, while processes related to the assistant's operations could take place on either device.