Stability AI Releases Stable Video 3D, an AI Model That Can Render 3D Videos From 2D Images

Stability AI released a new 3D video rendering model dubbed Stable Video 3D (SV3D) on Monday. The artificial intelligence (AI) model generates videos, but unlike popular video generators such as OpenAI's Sora, Runway AI, and Pika 1.0, it does not take text inputs. The main focus of SV3D is to take the image input and turn the 2D photo into an orbital 3D model. The company has made the new AI model public for commercial as well as non-commercial usage.

The announcement was made by the official account of Stability AI on X (formerly known as Twitter) in a post where it said, “Today, we are releasing Stable Video 3D, a generative model based on Stable Video Diffusion. This new model advances the field of 3D technology, delivering greatly improved quality and multi-view.” The announcement came just a month after the AI firm announced Stable Diffusion 3, which improves performance in multi-subject prompts.

Stable Video 3D AI model is being made available in two different variants — SV3D_u and SV3D_p. The former is capable of generating orbital videos based on single image inputs but it does not use camera conditioning. This means that while the objects in the 2D image will be converted into 3D renders, there will not be any camera movement. The more capable variant is SV3D_p which accommodates both single images and orbital views which will also allow it to create fully rendered 3D videos along specified camera paths.

As per the company, the AI model solves the inconsistency issues faced by older generation models such as Stable Zero123. SV3D leverages Neural Radiance Fields (NeRF) and mesh representations to improve the quality and consistency of rendered videos. “Additionally, in order to reduce the issue of baked-in lighting, Stable Video 3D employs a disentangled illumination model that is jointly optimized along with 3D shape and texture,” Stability AI said in a detailed blog post.

The Stable Video 3D is now available for both commercial and non-commercial usage. For commercial usage, users will require a Stability AI membership which starts at $20 (roughly Rs. 1650) a month for the Professional tier. For non-commercial usage, users can download the model weights on Hugging Face.