Tomorrow, will we all be 3D modelers? Nvidia has unveiled a generative AI, called LATTE3D, which instantly transforms text into 3D representations. The demonstration focused on objects and animals, but the tool could generate anything in 3D.
3D modeling specialists better watch out. Their activity could soon be challenged by a somewhat special artificial intelligence (AI) model. Indeed, LATTE3D is capable of converting written instructions on the fly (like on ChatGPT) into 3D representations of objects or animals.
This technology was unveiled on March 21, 2024 by Nvidia, and further illustrates the colossal investments of the American company in the field of AI – like the presentation of the Blackwell B200 chip, described as decisive for generative AI. These announcements took place during Nvidia’s GTC conference.
Concerning LATTE3D, the tool could be of interest to be used to quickly fill and populate three-dimensional virtual environments. This includes video games, advertising campaigns, location projections, planning tools for urban planning or digital training spaces — which AI could also take advantage of.
In a demonstration video, Nvidia presents prompts that result in 3D rendering. A turtle, a bird, a cat on a skateboard or even a coffee. The visuals proposed are certainly not photorealistic, but the shapes and proportions are respected, the work relatively fine, the texture and color effects present.
The amigurimi bird thus takes up this impression of crochet work, with large visible stitches. The origami cat includes the effects of folding with paper. The prompts presented on screen, however, are very basic. Operation on longer and more complex prompts remains to be discovered.
Generate 3D in moments
If the quality of the rendering will be appreciated differently, Nvidia nevertheless insists on another advantage of LATTE3D: its speed. “ A year ago, AI models took an hour to generate 3D images of this quality, while the current state of the art is 10 to 12 seconds “, according to Sanja Fidler, vice president of AI research at Nvidia.
With this tool, described as “ a virtual 3D printer “, it is possible to ” produce results much faster, putting near real-time 3D text generation within reach of creators in all industries », she adds. Above all, it opens the door to those new to modeling.
Above all, such a tool is supposed to better reflect the thinking of the modeler. Instead of browsing a library of 3D resources hoping to find the representation that best suits the project, you might as well create it from A to Z, describing it precisely. Design time is no longer so much of a concern. Only the accuracy of the prompt matters.
Nvidia’s demonstrations focused on animals and everyday objects, but its performance is not limited to these two categories. LATTE3D can easily handle other requests, if it has been previously trained with the appropriate datasets. Plants, cars, furniture, etc.
A prompt gives several proposals for different 3D shapes. It is then up to the Internet user to choose the rendering that they like best – we find this operation on Midjourney, for example, when it comes to generating images. Once a rendering is selected, it can be improved — but this may take a few minutes.
On the technical side, Nvidia says LATTE3D was trained using Nvidia A100 Tensor Core GPUs. Prompt management has been optimized with ChatGPT, “ in order to improve your ability to manage different sentences » from a user to describe a particular 3D object. Then, the generation can be done on the Nvidia RTX A6000 GPU.
However, this type of equipment is not within everyone’s reach — on merchant sites, the card is sold for several thousand euros. Too expensive for anyone who would like to get hold of this text-3D generative AI model, which is clearly still aimed at a professional and specialist audience. But in a few years?
If you liked this article, you will like the following: don’t miss them by subscribing to Numerama on Google News.