3D-GPT Creates 3D Worlds for Virtual Production Using Text Descriptions

Researchers at the University of Oxford, Australian National University, and Beijing Academy of Artificial Intelligence have introduced '3D-GPT,' an AI system capable of transforming textual descriptions into 3D models. This innovation, featured in a VentureBeat article and initially detailed in an arXiv paper, streamlines the creation of 3D content, offering an alternative to the complex and time-consuming methods traditionally associated with 3D modelling.

3D-GPT is designed to break down procedural 3D modelling tasks into manageable segments, assigning the most suitable AI agent to each task. This collaborative effort among multiple AI agents ensures the system's ability to interpret text prompts accurately and execute modelling functions effectively.

This technology holds promising potential for virtual production, particularly in crafting detailed backgrounds and environments, which traditionally require substantial time and effort. In tests, 3D-GPT demonstrated its prowess by generating complete 3D scenes from descriptions as vivid as “a misty spring morning, where dew-kissed flowers dot a lush meadow surrounded by budding trees,” producing realistic graphics that mirror the described elements accurately.

The system consists of three key agents: the task dispatch agent, the conceptualisation agent, and the modelling agent. These agents work together to achieve two primary objectives. Firstly, they systematically refine brief scene descriptions into detailed, dynamic designs, adjusting the text as needed based on further instructions. Secondly, they facilitate procedural generation by extracting parameter values from the enhanced text, allowing for smooth integration with 3D modelling software for efficient asset creation.

Source: Inavate