Search

Microsoft Phi-3.5 Open-Source AI Models Released, Claimed to Outperform Gemini 1.5 Flash, GPT-4o Mini

Microsoft has released Phi-3.5 MoE, Mini, and Vision AI models.

Advertisement
Highlights
  • All three AI models are available on Hugging Face
  • The Phi-3.5 MoE supports 1,28,000 tokens context length
  • The Phi-3.5 Vision has 4.2B parameters and an image encoder
Microsoft Phi-3.5 Open-Source AI Models Released, Claimed to Outperform Gemini 1.5 Flash, GPT-4o Mini

Running the Microsoft Phi-3.5 MoE model was trained on the Nvidia H100 80GB GPU

Photo Credit: Reuters

Microsoft released the family of Phi-3.5 artificial intelligence (AI) models on Tuesday, as the successor of the Phi-3 models introduced in April. The new release comprises Phi-3.5 Mixture of Experts (MoE), Phi-3.5 Vision, and Phi-3.5 Mini models. These are instruct models, so they will not work as a typical conversational AI but will require users to add specific instructions to get the desired output. The open-source AI models are available to download from the tech giant's Hugging Face listings.

Microsoft Releases Phi-3.5 AI Models

The release of the new AI models was announced by Microsoft executive Weizhu Chen in a post on X (formerly known as Twitter). The Phi-3.5 models offer upgraded capabilities over the predecessor, but the architecture, dataset and training methods largely remain the same. The Mini model has been updated with multilingual support, and the MoE and Vision models are new inclusions in the AI model family.

Coming to technicalities, the Phi-3.5 Mini has 3.8 billion parameters. It uses the same tokeniser (a tool that breaks down text into smaller units) and a dense decoder-only transformer. The model only supports text as input and supports a context window of 1,28,000 tokens. The company claims it was trained using 3.4 trillion tokens between June and August, and its knowledge cut-off is October 2023.

One key highlight of this model is that it now supports multiple new languages including Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, and Ukrainian.

The Phi-3.5 Vision AI model has 4.2 billion parameters and it includes an image encoder that allows it to process information within an image. With the same context length as the Mini model, it accepts both text and images as input. It was trained between July and August on 500 billion tokens of data and has a text knowledge cutoff of March.

Finally, the Phi-3.5 MoE AI model has 16x3.8 billion parameters. However, only 6.6 billion of them are active parameters when using two experts. Notably, MoE is a technique where multiple models (experts) are trained independently and then combined to improve the accuracy and efficiency of the model. This model was trained on 4.9 trillion tokens of data between April and August, and it has a knowledge cutoff date of October 2023.

On performance, Microsoft shared benchmark scores of all of the individual models, and based on the data shared, the Phi-3.5 MoE outperforms both Gemini 1.5 Flash and GPT-4o mini in the SQuALITY benchmark which tests the readability and accuracy when summarising a long block of text. This tests the long context window of the AI model.

However, it should be mentioned that it is not a fair comparison since MoE models use a different architecture and require more storage space and more sophisticated hardware to run. Separately, the Phi-3.5 Mini and Vision models have also outperformed relevant competing AI models in the same segment in some metrics.

Those interested in trying out the Phi-3.5 AI models can access them via Hugging Face listings. Microsoft said that these models use flash attention which will require users to run the systems on advanced GPUs. The company has tested them on Nvidia A100, A6000, and H100 GPUs.

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

 
Show Full Article
Please wait...
Advertisement

Related Stories

Popular Mobile Brands
  1. OnePlus Nord 5, OnePlus Nord CE 5 Launched in India at These Prices
  2. AI+ Pulse, AI+ Nova 5G With 50-Megapixel Rear Cameras Launched in India
  3. AI+ Nova 5G, Pulse Phones India Launch Today: How to Watch Live Event
  4. OnePlus Nord 5 Review
  5. Apple Releases iOS 26 Beta 3 Update for iPhone With These New Features
  6. OnePlus Buds 4 With Up to 45 Hours of Total Battery Life Launched in India
  7. OnePlus 13 to Get a Price Cut During Amazon Prime Day 2025 Sale
  8. OnePlus Nord 5, Nord CE 5 Launch Today: Everything You Need to Know
  9. Realme 15 5G, 15 Pro 5G to Launch in India on This Date
  10. Samsung Galaxy S25 FE Said to Get a Flexible OLED Display This Year
  1. Amazon Prime Day 2025 Sale: Samsung Galaxy Buds 3 Pro to Be Available at a Discounted Price
  2. Oppo Reno 14 Launched in New Finish With Temperature-Sensitive Colour Changing Rear Panel
  3. Microsoft Edge Can Now Load Websites Faster After Migration to WebUI 2.0, Says Company
  4. Samsung Galaxy S25 FE to Sport a 6.7-Inch Flexible OLED Display: Report
  5. ChatGPT Could Soon Get a Study Together Feature That Helps Users Learn New Topics
  6. OnePlus Buds 4 Launched in India With Adaptive ANC, Up to 45 Hours of Total Battery Life: Price, Features
  7. Amazon Prime Day 2025 Sale: OnePlus 13 Offer Price Revealed
  8. Ai+ Wearbuds Smartwatch With Built-In TWS Earbuds, Blood Pressure Monitoring Launched in India
  9. Apple Loses Top AI Models Executive to Meta’s Hiring Spree
  10. OnePlus Nord 5, Nord CE 5 Launched in India With 50-Megapixel Rear Camera, OnePlus AI Features
Gadgets 360 is available in
Download Our Apps
App Store App Store
Available in Hindi
App Store
© Copyright Red Pixels Ventures Limited 2025. All rights reserved.
Trending Products »
Latest Tech News »