Google Reportedly Rolling Out Gemini Live Video and Screen-Sharing Features

Google is reportedly rolling out its two major Gemini features — live video and screen-sharing. The Mountain View-based tech giant first unveiled these features at Google I/O 2024. Developed by Google DeepMind under Project Astra, these features come with live multimodal data processing capabilities and allows the artificial intelligence (AI) chatbot to answer queries about the user's device and their surroundings in real-time. The company had previously said that these new features will be rolling out by March. Notably, currently these features are only available to Gemini Advanced subscribers on the mobile apps.

Google Reportedly Rolling Out New Gemini Features

First spotted by 9to5Google, Reddit user Kien_PS recently posted a screenshot on the Bard (Gemini's older name) subreddit showcasing the “Share-screen with live” feature. The same user again posted a demo video of the feature on Sunday, highlighting how it works.

Separately, Google spokesperson Alex Joseph told The Verge that the new AI features are rolling out to Gemini Live. Apart from screen-sharing, Gemini will also be able to access the user's device's camera and answer queries about whatever the user sees in real-time.

This real-time data processing capability will now allow users to ask Gemini queries about outfit suggestions by showing it their wardrobe, or identifying a monument or a store when outdoors. The screen-sharing feature, which is an enhanced version of the existing “Talk about the screen” feature will let Gemini help the user as they navigate across various screens on their smartphone.

Both of these features are part of Gemini Live which was rolled out to users last year, and can have a two-way live voice conversation with users. Google had previously said that it wants to make Gemini more useful in real-time situations.

Notably, the Gemini Live video feature is similar to OpenAI's Advanced Voice Mode with Vision feature for ChatGPT and the real-time video feature in Ray-Ban Meta Smart Glasses. As AI and the underlying infrastructure behind the technology advances, and cloud servers get more capable, tech giants can now offer faster inference for real-time use cases.

Notably, these two Gemini features are only available to the Gemini Advanced subscribers currently. The company has not shared any information on when and if it will be expanded to the free tier. Gemini Advanced subscription can be purchased as part of the Google One AI Premium plan at the price of Rs. 1,950.