Apple Vision Pro Gets Apple Intelligence Capabilities and New Features With visionOS 2.4 Update

Apple released the visionOS 2.4 update for the Apple Vision Pro on Monday. It is considered a major update and introduces the first set of Apple Intelligence features for the mixed reality (MR) headset. According to the Cupertino-based technology giant, users can now leverage Writing Tools for summarising and editing text, Image Playground for reimagining photos, and natural language search in Photos to find the best memories quickly. Further, the update also bundles a new feature called Spatial Gallery which brings curations by Apple.

visionOS 2.4 Update for the Apple Vision Pro: Features

Apple detailed the new features part of the visionOS 2.4 update in a newsroom post and the features are available on the headset with the device and Siri language set to English (US). The iPhone maker says Writing Tools is now available on the Apple Vision Pro. It is an AI-powered feature with options for composing, rewriting and proofreading documents with spelling and grammar checks. It can also summarise the text, create lists and tables, or highlight the key points from the text.

The update brings Image Playground, a standalone app which leverages generative AI to create images based on descriptive prompts. Users can select a reference person from their Photos app for inspiration and can choose from different art styles. The same app also bundles Genmoji which works similarly but for emojis. Apple says these can be added inline to messages, shared as a sticker, or sent as a Tapback.

Apple Intelligence now allows users to search for specific images stored in the Photos app with natural language queries. Further, the Create a Memory Movie enables AI to craft a storyline with chapters based on themes identified from the photos, and arrange them into a movie with a narrative arc, depending upon the prompt entered by the user. Meanwhile, Smart Reply in Mail and Messages allows for quick replies.

Apple says visionOS 2.4 also brings support for Priority Messages in Mail, Mail Summaries, Image Wand in Notes, Priority Notifications in Notification Centre, and Notification Summaries.

The company emphasises that photos and videos used to create movies and other actions are kept private and not shared with the company. Additionally, Apple Intelligence uses on-device processing whenever possible to protect user privacy. For tasks that require larger AI models, Apple engages Private Cloud Compute and the users' data is never stored or shared with the company.