Stability AI and Arm Release Lightweight Tex-to-Audio Model Optimised for Fast On-Device Generation

The new text-to-audio AI model developed by Stability AI and Arm is called Stable Audio Open Small.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 15 May 2025 12:24 IST

Highlights

Stability AI has open-sourced the Stable Audio Open Small AI model
It is a 341 million parameter text-to-audio model
Stable Audio Open Small can produce up to 11 seconds of audio

Stability AI and Arm Release Lightweight Tex-to-Audio Model Optimised for Fast On-Device Generation

Stability AI says the text-to-audio model can run locally on a smartphone

Photo Credit: Stability AI

Stability AI developed a new text-to-audio generation artificial intelligence (AI) model in partnership with Arm. Announced on Wednesday, the new model is dubbed Stable Audio Open Small, and it is said to generate short audio samples using text prompts. The London-based AI firm said that the model is lightweight and is optimised to run entirely on Arm CPUs. It is also said to have a fast generation time, making it useful for bulk use cases. The open-source audio model is available to download from GitHub and Hugging Face.

Stability AI Releases Stable Audio Open Small

In a newsroom post, the AI firm detailed the new large language model. It is a distilled version of the Stable Audio Open model, which was released in June 2024, and can generate up to 47 seconds of audio. The smaller text-to-audio model was designed with a focus on faster generation speed and smaller size.

The Stable Audio Open Small is a 341 million parameter model that can generate up to 11 seconds of audio. The company claims that it can generate an audio sample in less than eight seconds while running locally on a smartphone. Interestingly, Stability AI and Arm announced their collaboration for generative audio creation at Mobile World Congress (MWC) 2025.

Apple's New Matrix3D Model Can Turn Flat Images Into Dynamic 3D Scenes

Coming to the architecture and training, the Stable Audio Open Small is a latent diffusion model based on a transformer architecture. It is trained on a dataset of 4,86,492 audio recordings. The company said that all audio files are licensed. For text conditioning, a publicly available pre-trained T5 model was used. The AI firm used the Adversarial Relativistic-Contrastive (ARC) algorithm in the post-training phase to improve prompt adherence and increase the inference speed.

As per the company, this text-to-audio model is suited for creating drum loops, foley, instrument riffs, and ambient textures. Due to its small size, it can be deployed on Arm-powered smartphones as well as edge devices. The model can also be used in scenarios where real-time generation and responsiveness matter.

Stable Audio Open Small's model weights can be downloaded on the AI firm's Hugging Face listing, and the code base can be found on the GitHub listing. The AI model is available for commercial and non-commercial use under the permissive Stability AI Community Licence.