Apple Researchers Working on MM1, a Family of Multimodal AI Model With Up to 30 Billion Parameters

Apple researchers said that the MM1 AI models are currently in the pre-training phase.

Advertisement
Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 18 March 2024 13:04 IST
Highlights
  • Apple researchers said MM1 has achieved competitive performance
  • The paper claims MM1 is capable of in-context learning
  • Recently, Apple acquired AI startup DarwinAI
Apple Researchers Working on MM1, a Family of Multimodal AI Model With Up to 30 Billion Parameters

Apple researchers said the MM1 model consists of both dense models and MoE variants

Photo Credit: Unsplash/Laurenz Heymann

Apple researchers have shared their work on building a multimodal artificial intelligence (AI) large language model (LLM), in a pre-print paper. Published on an online portal on March 14, the paper highlights how it was able to achieve the advanced capabilities of multimodality and make the foundation model train on both text-only data as well as images. The new advancements in AI for the Cupertino-based tech giant come following CEO Tim Cook's remarks made during the company's earning calls where he said that AI features could arrive later this year.

The pre-print version of the research paper has been published on arXiv, an open-access online repository of scholarly papers. However, the papers posted here are not peer-reviewed. While the paper itself does not mention Apple, most of the researchers mentioned are affiliated with the company's machine learning (ML) division, leading to the belief that the project is also affiliated with the iPhone maker.

As per the researchers, they are working on MM1, a family of multimodal models containing up to 30 billion parameters. Calling it a “performant multimodal LLM (MLLM), the authors of the paper highlighted that image encoders, the vision language connector, and other architecture components and data choices were made to create the AI model which is capable of understanding both text as well as image-based inputs.

Giving an example, the paper stated, “We demonstrate that for large-scale multimodal pre-training using a careful mix of image-caption, interleaved image-text, and text-only data is crucial for achieving state-of-the-art (SOTA) few-shot results across multiple benchmarks, compared to other published pre-training results.”

Advertisement

To break it down, the AI model is currently in the pre-training phase, which means it is not trained enough to give the desired outputs. This is the stage when the algorithm and the AI architecture are used to design the workflow of the model and how it processes data, eventually. The team of Apple researchers were able to add computer vision to the model using image encoders and a vision language connector. Then, when testing with a mix of just images, image and text, and text-only data set, the team found that the results were competitive compared to existing models at the same stage.

While the breakthrough is significant, this research paper is not enough to ascertain that a multimodal AI chatbot will be added to Apple's operating system. At this stage, it is difficult to even say whether the AI model is multimodal while taking inputs or in giving output as well (whether it can generate AI images or not). But if the results are confirmed to be consistent after peer review, it can be said that the tech giant has taken another big step towards building a native generative AI foundation model.


Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.
Affiliate links may be automatically generated - see our ethics statement for details.
 

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Apple, Artificial intelligence, AI
Advertisement

Related Stories

Popular Mobile Brands
  1. OnePlus 13 Series Gets AI-Powered Plus Mind Feature With Latest Update
  2. Vivo X200 FE Launched in India With 6,500mAh Battery, Compact Design
  3. Realme 15 Pro 5G Display, Battery Details Confirmed Ahead of India Launch
  4. Samsung Galaxy F36 5G to Launch Soon in India: Details
  5. Google Pixel 10 Pro Fold May Offer Larger Cover Display, Faster Charging
  6. Nothing Phone 3 and Headphone 1 to Go on Sale in India From Today
  7. Amazon Prime Day Sale 2025 Highlights: Top Last-Minute Deals on Day 3
  8. Samsung Galaxy S25 FE Key Specifications Leaked: All Details
  9. Up to 70 Percent Discount on Headphones During Amazon Prime Day Sale 2025
  1. Tesla Model Y Price in India Announced; Tesla App for iOS Available to Download
  2. iPhone 17 Components Arrive in India; Mass Production Expected to Begin in August: Report
  3. Vivo Y50 5G, Vivo Y50m 5G Appear on China Telecom Site With MediaTek Dimensity 6300 SoC, 6,000mAh Battery
  4. Google Pixel 10 Pro Fold Said to Get Larger Battery, Faster Charging and Bigger Cover Display
  5. Nothing Phone 3 and Headphone 1 Sale in India Begins Today: Price, Offers
  6. OnePlus 13 Series Update Brings Plus Mind Feature for Quickly Storing Information in Mind Space
  7. Earth’s Spin to Speed Up Briefly, Causing Shorter Days This Summer
  8. James Webb Telescope Spots Rare ‘Cosmic Owl’ Formed by Colliding Galaxies
  9. MIT Develops Low-Resource AI System to Control Soft Robots with Just One Image
  10. Ax-4 Astronauts to Return from ISS with 580 Pounds of Science Cargo
Gadgets 360 is available in
Download Our Apps
Available in Hindi
© Copyright Red Pixels Ventures Limited 2025. All rights reserved.