Search

Google I/O 2024: DeepMind Showcases Real-Time Computer Vision-Based AI Interaction With Project Astra

Google’s Project Astra showed an AI chatbot using a smartphone camera to describe the environment in real time.

Advertisement
Highlights
  • In the demo, the AI could identify objects and settings
  • The AI chatbot could also remember what it saw previously
  • Google said it is a demo of the capabilities of future AI assistants
Google I/O 2024: DeepMind Showcases Real-Time Computer Vision-Based AI Interaction With Project Astra

Showing the demo, Demis Hassabis, CEO of Google DeepMind, said it was shot in a single take in real-time

Photo Credit: Google

Google I/O 2024's keynote session allowed the company to showcase its impressive lineup of artificial intelligence (AI) models and tools that it has been working on for a while. Most of the introduced features will make their way to public previews in the coming months. However, the most interesting technology previewed in the event will not be here for a while. Developed by Google DeepMind, this new AI assistant was called Project Astra and it showcased real-time, computer vision-based AI interaction.

Project Astra is an AI model that can perform tasks that are extremely advanced for the existing chatbots. Google follows a system where it uses its largest and the most powerful AI models to train its production-ready models. Highlighting one such example of an AI model which is currently in training, the co-founder and CEO of Google DeepMind Demis Hassabis showcased Project Astra. Introducing it, he said, “Today, we have some exciting new progress to share about the future of AI assistants that we are calling Project Astra. For a long time, we wanted to build a universal AI agent that can be truly helpful in everyday life.”

Hassabis also listed a set of requirements the company had set for such AI agents. They need to understand and respond to the complex and dynamic real-world environment, and they need to remember what they see to develop context and take action. Further, it also needs to be teachable and personal so it can learn new skills and have conversations without delays.

With that description, the DeepMind CEO showcased a demo video where a user could be seen holding up a smartphone with its camera app open. The user speaks with an AI and the AI instantly responds, answering various vision-based queries. The AI was also able to use the visual information for context and answer related questions required generative capabilities. For instance, the user showed the AI some crayons and asked the AI to describe it with alliteration. Without any lag, the chatbot says, “Creative crayons colour cheerfully. They certainly craft colourful creations.”

But that was not all. Further in the video, the user points towards the window, from which some buildings and roads can be seen. When asked about the neighbourhood, the AI promptly gives the correct answer. This shows the capability of the AI model's computer vision processing and the massive visual dataset it would have taken to train it. But perhaps the most interesting demonstration was when the AI was asked about the user's glasses. They appeared on the screen briefly for a few seconds and it had already left the screen. Yet, the AI could remember its position and guide the user to it.

Project Astra is not available either in public or private preview. Google is still working on the model, and it has to figure out the use cases for the AI feature and decide how to make it available to users. This demonstration would have been the most ridiculous feat by AI so far, but OpenAI's Spring Update event a day ago took away some of its thunder. During its event, OpenAI unveiled GPT-4o which showcased similar capabilities and emotive voices that made the AI sound more human.

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

 
Show Full Article
Please wait...
Advertisement

Related Stories

Popular Mobile Brands
  1. Redmi Note 14 SE 5G Launched in India With 50-Megapixel Rear Camera
  2. Vi Non-Stop Hero Prepaid Plans Now Available in These Circles
  3. OnePlus Buds 4 Review: The New Standard for Value ANC Earbuds
  4. iPhone 16e Available at a Discounted Price in India: See Offer
  5. Oppo K13 Turbo Series Confirmed to Launch in India Soon
  6. Vivo V60 to Launch in India Soon; Reportedly Spotted on TRDA Site
  1. Doomed Exoplanet TOI-2109b Spirals Toward Its Star with Three Possible Fates
  2. Hubble Captures First Images of Ancient Interstellar Visitor Comet 3I/ATLAS
  3. Meet Walker S2: The Self-Sufficient Humanoid Robot with Swappable Battery Tech
  4. Saare Jahan Se Acha OTT Release Date: When and Where to Watch Pratik Gandhi Starrer Movie
  5. The Plot Now Streaming on Lionsgate Play: What You Need to Know About Korean Crime Thriller Movie
  6. Hari Hara Veera Mallu OTT Release Reportedly Revealed: Everything You Need to Know
  7. X&Y Now Streaming on SunNXT: What You Need to Know About This Comedy Kannada Drama Movie
  8. Parijatha Parvam Streaming Now on LionsgatePlay: A Twisted Telugu Crime Comedy Full of Chaos and Laughter
  9. Mortal Kombat: Legacy Kollection's Leaked Release Date Points to September Launch
  10. Primebook 2 Neo With MediaTek Helio G99 SoC to Launch in India on July 31; Price, Specifications Teased
Gadgets 360 is available in
Download Our Apps
App Store App Store
Available in Hindi
App Store
© Copyright Red Pixels Ventures Limited 2025. All rights reserved.
Trending Products »
Latest Tech News »