3D-ChatGPT can create 3D worlds for virtual production using just text descriptions

Researchers from the University of Oxford, the Australian National University,and the Beijing Academy of Artificial Intelligence have developed a new AI system called '3D-GPT' that can generate 3D models from text-based descriptions provided by a user.
 

The story, which was picked up by VentureBeat, describes the system, published on arXiv originally, which offers a more efficient and intuitive way to create 3D content compared to traditional 3D modeling workflows.

3D-GPT is able to “dissect procedural 3D modeling tasks into accessible segments and appoint the apt agent for each task,” according to the paper. It uses multiple AI agents that focus on a different part of understanding the text prompt to execute modeling functions.

The application could have significant applications in virtual production, where creating backgrounds and environments requires a lot of time-consuming effort. 

The system was tested on prompts like “a misty spring morning, where dew-kissed flowers dot a lush meadow surrounded by budding trees.” 3D-GPT was able to generate complete 3D scenes with realistic graphics that accurately reflected elements described in the text.

3D-GPT comprises three pivotal agents: task dispatch agent, conceptualisation agent, and modeling agent. Together, they collaboratively pursue two essential goals. First, it systematically enhances concise initial scene descriptions, evolving them into intricate forms while dynamically adapting the text based on subsequent instructions. Second, it seamlessly integrates procedural generation, extracting parameter values from enriched text to effortlessly interface with 3D software for asset creation.

 

Source: Inavate