Microsoft Phi-3.5 Open-Source AI Models Released, Claimed to Outperform Gemini 1.5 Flash, GPT-4o Mini

Microsoft has released Phi-3.5 MoE, Mini, and Vision AI models.

Advertisement
Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 21 August 2024 13:18 IST
Highlights
  • All three AI models are available on Hugging Face
  • The Phi-3.5 MoE supports 1,28,000 tokens context length
  • The Phi-3.5 Vision has 4.2B parameters and an image encoder
Microsoft Phi-3.5 Open-Source AI Models Released, Claimed to Outperform Gemini 1.5 Flash, GPT-4o Mini

Running the Microsoft Phi-3.5 MoE model was trained on the Nvidia H100 80GB GPU

Photo Credit: Reuters

Microsoft released the family of Phi-3.5 artificial intelligence (AI) models on Tuesday, as the successor of the Phi-3 models introduced in April. The new release comprises Phi-3.5 Mixture of Experts (MoE), Phi-3.5 Vision, and Phi-3.5 Mini models. These are instruct models, so they will not work as a typical conversational AI but will require users to add specific instructions to get the desired output. The open-source AI models are available to download from the tech giant's Hugging Face listings.

Microsoft Releases Phi-3.5 AI Models

The release of the new AI models was announced by Microsoft executive Weizhu Chen in a post on X (formerly known as Twitter). The Phi-3.5 models offer upgraded capabilities over the predecessor, but the architecture, dataset and training methods largely remain the same. The Mini model has been updated with multilingual support, and the MoE and Vision models are new inclusions in the AI model family.

Coming to technicalities, the Phi-3.5 Mini has 3.8 billion parameters. It uses the same tokeniser (a tool that breaks down text into smaller units) and a dense decoder-only transformer. The model only supports text as input and supports a context window of 1,28,000 tokens. The company claims it was trained using 3.4 trillion tokens between June and August, and its knowledge cut-off is October 2023.

One key highlight of this model is that it now supports multiple new languages including Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, and Ukrainian.

Advertisement

The Phi-3.5 Vision AI model has 4.2 billion parameters and it includes an image encoder that allows it to process information within an image. With the same context length as the Mini model, it accepts both text and images as input. It was trained between July and August on 500 billion tokens of data and has a text knowledge cutoff of March.

Finally, the Phi-3.5 MoE AI model has 16x3.8 billion parameters. However, only 6.6 billion of them are active parameters when using two experts. Notably, MoE is a technique where multiple models (experts) are trained independently and then combined to improve the accuracy and efficiency of the model. This model was trained on 4.9 trillion tokens of data between April and August, and it has a knowledge cutoff date of October 2023.

Advertisement

On performance, Microsoft shared benchmark scores of all of the individual models, and based on the data shared, the Phi-3.5 MoE outperforms both Gemini 1.5 Flash and GPT-4o mini in the SQuALITY benchmark which tests the readability and accuracy when summarising a long block of text. This tests the long context window of the AI model.

However, it should be mentioned that it is not a fair comparison since MoE models use a different architecture and require more storage space and more sophisticated hardware to run. Separately, the Phi-3.5 Mini and Vision models have also outperformed relevant competing AI models in the same segment in some metrics.

Advertisement

Those interested in trying out the Phi-3.5 AI models can access them via Hugging Face listings. Microsoft said that these models use flash attention which will require users to run the systems on advanced GPUs. The company has tested them on Nvidia A100, A6000, and H100 GPUs.

 

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Advertisement

Related Stories

Popular Mobile Brands
  1. OTT Releases This Week: Pattth, Stolen, Jaat, Bhool Chuk Maaf, and More
  2. Nothing Phone 3 Rear Panel Revealed in New Teaser Ahead of June 1 Launch
  3. Motorola Edge 60 Will Launch in India on This Date
  4. iQOO Z10 Lite 5G India Launch Date, Design and Battery Size Confirmed
  5. iPhone 17 May Support 50W Wireless Charging With New MagSafe Chargers
  1. Google Chrome Gets 'Highest Ever' Speedometer Score; Company Reveals Optimisations Behind Improved Performance
  2. Nothing Phone 3 Rear Panel Design Teased; Suggests Departure From Signature Glyph Design
  3. Xiaomi Smart Band 10 Leaked Marketing Images Suggest Design and Key Features
  4. 'We're Not Done Yet': CD Projekt Red Confirms Cyberpunk 2077 Is Getting Another Update Later This Month
  5. Microsoft Introduces Copilot Shopping With Native Checkout Capability in App
  6. Vivo Y-Series Smartphone With Curved Display Said to Launch in India; Colour Options Leaked
  7. Uber Reportedly Exploring Stablecoin Adoption to Cut Cross-Border Transfer Costs
  8. Perplexity AI CEO Pans Google’s AI Rollout and Assistant Product
  9. Tecno Pova 7 Neo 4G Design Spotted in Leaked Hands-On Images; Key Features Surface Online
  10. PhonePe to Launch UPI Payments App for Feature Phones With P2P Transfers, Offline QR Payments
Gadgets 360 is available in
Download Our Apps
Available in Hindi
© Copyright Red Pixels Ventures Limited 2025. All rights reserved.