Researchers Unveil LegoGPT AI Model That Can Build Physically Stable Design of Lego Structures

LegoGPT is essentially a text-to-3D rendering AI model, where the generated structures can also be made in the real world.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 12 May 2025 12:38 IST

Highlights

LegoGPT is based on a fine-tuned version of LLaMA-3.2-Instruct-1B
The AI model was trained on a dataset of over 47,000 Lego structures
LegoGPT is an open-source AI model available on GitHub

Researchers Unveil LegoGPT AI Model That Can Build Physically Stable Design of Lego Structures

In case a Lego structure is not stable, the model can roll back to the unstable brick

Photo Credit: LegoGPT

LegoGPT, a new artificial intelligence (AI) model that can generate three-dimensional (3D) Lego structure designs, was unveiled by researchers recently. The new AI model is an open-source project to determine whether AI models can generate structures that are consistent with real-world physics and are consistently stable. The researchers have shared details of how the model was built, as well as have made the dataset available to the public domain. The AI-generated Lego structures were also tested by humans and robots to confirm the stability of the structures.

LegoGPT AI Model Was Built on LLaMA-3.2-Instruct

In a post, researchers from Carnegie Mellon University detailed the LegoGPT AI model. The large language model (LLM) can generate a Lego structure from a text prompt, ensuring it is physically stable and buildable. The open-source model is available to download and use on GitHub with a permissive MIT licence.

Users can prompt the model to design a “streamline elongated vessel” or a “backless bench with armrest,” and it can generate a design that not only matches the description but also can be placed upright without the structure collapsing.

Baidu Looks to Patent AI System to Decipher Animal Sounds

This is possible due to two components that make LegoGPT — the base AI model and a stability analysis system. For the base model, the researchers used a fine-tuned version of the Llama-3.2-Instruct with one billion parameters. This was paired with Gurobi, a mathematical optimisation solver, that runs stability analysis for each generated structure.

Alongside building the refined architecture, the researchers also created a dataset to train the model on Lego structures. Dubbed StableText2Lego, it is a dataset containing more than 47,000 Lego structures of over 28,000 unique 3D objects. Each structure is accompanied by detailed captions, design code, and models.

To verify that the generated structures are indeed stable, the researchers also tested them with a dual robot assembly. The assembly was tasked with recreating the designs and testing whether they could stand upright. Some of the designs were also recreated by humans to see the impact on stability if less dextrous hands were involved. The research paper claims that 99.8 percent of all structures passed the stability test.

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.