Google Upgrades Gemini Live With Visual Guidance, Adds Support for Phone and Messages Apps

Google announced a new update for Gemini Live on Wednesday, adding new capabilities, broader app integration, and improved voice models for more natural speech. The Mountain View-based tech giant is also introducing visual guidance, allowing Gemini Live to highlight specific objects on the screen in response to user queries. In addition, the real-time, two-way voice chatbot is being integrated with the Phone, Messages, and Clock apps, enabling it to make calls, send messages, and set alarms. These features will first roll out to the Google Pixel 10 series.

Gemini Live Gets a Major Upgrade

In a blog post, the tech giant detailed the new features being added to Gemini Live. Notably, visual guidance will be available on the Pixel 10 series on August 28, the day the smartphones go on sale. The same week, these features will also start rolling out to other Android devices. The iOS devices will receive the feature after that. The company did not mention any specific timeline for the app integration and voice model upgrades.

Visual guidance was first teased at this year's Google I/O event. Essentially, when a user turns on the video feed in Gemini Live and the visuals show multiple objects, the AI chatbot can highlight the object it is talking about. On the screen, it appears as a rectangle with white margins. The company says the feature can be useful in scenarios where the visual is crowded by multiple objects, or the user is seeking information about two visually similar objects.

Apart from this, the tech giant is also expanding the first-party app integration of Gemini Live. So far, it only supports Google Calendar, Keep and Tasks. However, the company says it is planning to add support for the Phone, Messages, and Clock apps as well. Once the feature rolls out, users can ask the AI to make a call, set an alarm, or send a message to a contact mid-conversation, without needing to stop the conversation.

Finally, the company announced that it will soon launch updates to the speech models that power Gemini Live. The update will improve the intonation, rhythm, and pitch to make the audio sound more natural and expressive. Gemini Live will also be able to change the tone of its voice based on the topic, adjust its tempo to speak slower or faster, and even use character accents while narrating a story.