Simple productions are generated in minutes. © Capture Point-E
ChatGPT, DALL-E and now Point-E. OpenAI is constantly developing online tools based on artificial intelligence. The organization founded by Elon Musk presented, on Tuesday, December 20, 2022, its new tool aimed at generating three-dimensional images using simple text.
An image generated in minutes
Usually, producing a 3D computer-generated image requires significant computing power and long hours of rendering. With the tool developed by OpenAI, it is possible to create an image in one or two minutes with a single GPU (graphics processing unit). To generate a work, a simple sentence in natural language is sufficient. Point-E’s models and code have just been published on OpenAI’s Github repository.
To generate fissa an image, Point-E will actually produce a synthetic view of the concept to be created. A cloud of points aligned according to the user’s request will be produced, delivering a global visual of the requested object. “Our method first generates a single synthetic view using a text-image diffusion model, then produces a 3D point cloud using a second diffusion model that conditions the generated image”details OpenAI.
The different steps that lead to the creation of a 3D image. © Capture OpenAI
To produce a correct image-to-text match, the text-to-image model of Point-E was trained by deep learning on “a large corpus of pairs (text, image)”. The image/3D model, on the other hand, is based on a smaller dataset of pairs, the researchers explain. The generation of 3D images in a few minutes is a real innovation that could be of interest to many sectors. A video adaptation of Point-E to produce effects, objects or even 3D characters is even possible.
Still low resolution
Currently the OpenAI model is based on synthetic renderings and does not allow optimal quality. “Furthermore, while our method produces colored three-dimensional shapes, it does so at a relatively low resolution in a 3D format (point clouds) which does not capture fine shapes or textures”say the researchers.
Several creations generated with Point-E. © Point-E / Github
In addition, the 3D images generated by Point-E could make it possible to manufacture objects in the real world using a 3D printer. Virtually creating a dangerous 3D object could then open the possibility of an impression of the latter, alert the specialists. We imagine for example the generation of a weapon in 3D by a simple textual command on the model.
For its part, OpenAI hopes that its approach “could serve as a starting point for further work in the field of 3D synthesis”.