Google Shares Developments for On-Device Generative AI Features in Android Devices

Google I/O 2024 shed light on the new artificial intelligence (AI) initiatives taken by the company and how it plans to integrate them within its Android platform. During sessions hosted for developers, the tech giant delved deeper into on-device AI developments that app makers will soon be able to tap into. It currently offers both server-based AI features as well as on-device AI features, but these are mainly utilised by Google's in-house apps. Notably, the company also announced an improved Google TalkBack service powered by Gemini Nano with multimodality later this year.

The 19-minute-long video titled “Android on-device AI under the hood” was posted on YouTube in Google's Android Developers channel. The developer-focused session opened with a discussion of the benefits of keeping AI processing localised within the device and its limitations. Some of the advantages highlighted include local processing, offline availability, potentially reduced latency, and no additional cost. Here, additional costs refer to cloud computing costs.

While not mentioned in the video, on-device AI is also a good thing for users in terms of privacy. When the computing occurs within the device, the data AI uses to generate content never leaves the device, and cannot be accessed by even Google or a third-party app developer. However, all these benefits come at a cost. The biggest is the limitation of the computational capability of a smartphone-sized device. Despite new chipsets being introduced with neural processing units (NPU) and more powerful GPUs, they cannot compete with the cloud-based infrastructure.

This means the AI features will be limited in what they can do. Highlighting the same, Google classified on-device AI features into three groups Consume, Create, and Classify. Consume includes actions such as summarising a page or providing an overview of an email. Create includes generating text and rephrasing and rewriting them, while Classify includes sentiment analysis and checking the tonality of a message. Notably, Google did not mention any multimodal AI features, likely because powering them locally on the device is not possible at present.

The session also focused on Gemini Nano, which appears to be the preferred choice for the tech giant for Android's on-device features. While Gemma was also mentioned as a potential alternative, most of Google's current on-device AI features such as Magic Compose in Google Message, Summarise in Pixel Recorder, and Gboard smart replies are all built on Nano. Further, the company has also confirmed that Google Talkback service will also get an upgrade later this year and will be powered by Gemini Nano.

So far, the selection of apps that use on-device AI is limited to Google's in-house apps. But the company is hoping that through these interactions, it can bring more developers into the foray.