Google Integrating Gemini AI Capabilities Into Nest Cameras and Google Home Experiences

Google announced Gemini AI integration into Google Home devices and Google Nest cameras on Tuesday. The artificial intelligence (AI) features will offer multimodal capabilities to these devices, such as better image and video segmentation and identification, improved natural conversation, and more. Alongside, the tech giant also announced the successor to the Chromecast with Google TV, which is now being rebranded as Google TV Streamer. The fourth generation of Google Nest Learning Thermostat was also announced by the company. The availability of the Nest is limited to the US and Canada.

Gemini Integration Comes to Google Home Devices

In a blog post, Google revealed that multimodal Gemini AI models were being integrated with the Google Home devices and Google Nest cameras to offer an AI-powered experience to users. The announcement comes after the tech giant opened its smart home platform platform at the Google I/O event this year.

Gemini AI in Google Nest cameras
Photo Credit: Google

The most notable upgrade will come to Google Nest security cameras. Powered by Gemini, these cameras will now offer video segmentation and identification. What this means is when the cameras see an activity, instead of offering a basic description, it will now be able to understand and contextualise more information. Citing an example, Google said Nest cameras will be tell the user that a “dog is digging in the garden” instead of just an “animal detected”.

Further, users can also check their camera activity by searching them in the Google Home app. This will allow users to check for things that have happened multiple days in the past, instead of checking the entire footage. Google says it will be possible to ask questions such as “Did the kids leave their bikes in the driveway?” or “Did the FedEx truck drive by today?” and the AI will quickly find the answer and share a helpful summary.

Automation via Google Home devices will also be powered by Gemini AI. Using a Google Home speaker or a Display hub will allow users to set up all smart devices with a specific function, even if the user does not want to go through the entire complicated automation process. Using the app or the display, users can type in what they want from automation as a simple text prompt and the AI will figure out the rest.

For instance, a user can say something like “Turn off all the lights and keep the AC at 24 degrees Celsius at bedtime” and the AI will find out the user's bedtime from past behaviour and adjust the lights and the AC accordingly.

Finally, Google Assistant within the Google Home speakers is also getting an upgrade. It will be able to hold more natural conversations and users will be able to go back and forth on a particular topic. Users will also be able to ask follow up questions seamlessly. Later this year, Google will also add more voices to the assistant that users can choose from.

The Gemini AI integration within Google Home and Nest cameras will be rolled out in public preview later this year to Nest Aware subscribers.