Taken by surprise in the AI race against ChatGPT or even Midjourney and Dall-E, Google has worked hard to catch up in this fierce competition. After the impressive presentation of Gemini, the Mountain View firm has just unveiled Lumière, an absolutely astonishing video generation AI.
In the race for artificial intelligence that the tech giants are currently leading, Google is somewhat behind. Let’s say that the Mountain View firm didn’t really see the ChatGPT phenomenon coming, followed by the explosion of AI-based consumer tools like Dall-E or Midjourney.
Never mind, the Alphabet subsidiary has worked hard to get back into the game, first of all with Bard, its conversational AI. But in December 2023, Google hit hard by presenting Gemini, a brand new artificial intelligence that is significantly more efficient than its main competitor: ChatGPT. Note that Gemini is already integrated into the Pixel 8 Pro, in the United States at least. In Europe, GDPR has slowed down the process.
Google presents Lumière, its absolutely mind-boggling video generation AI
But Google does not intend to stop there, quite the contrary. In fact, the web giant has just revealed Lumière, an AI dedicated to video generation. A much more arduous and complex task than generating images. For good reason, to generate a video from scratch, an AI must take multiple factors into account such as movement or possible interactions with the decor (collisions, difficult terrain, etc.).
It is also necessary to achieve a relatively fluid sequence, where the actions follow one another coherently. To do this, and rather than assembling a succession of images like in a cartoon, Lumière creates the video from A to Z thanks to simultaneous management of objects and their movement. “The U-Net Space-Time architecture generates the entire temporal duration of the video at once, through a single pass through the model. This contrasts with existing video models that synthesize distant keyframes followed by temporal super-resolution, an approach that makes overall temporal coherence inherently difficult,” explain the researchers behind the project.
To give us an idea of Light’s capabilities, scientist Hila Chefer shared some excerpts and demonstrations on X. Concretely, Lumière can generate videos of around 5 seconds in definition 1024 x 1024 pixels. To do this, it can either be based on textual command lines, but also from an image. It can also animate certain parts of still images (like the smoke from a locomotive for example as we can see in the video above). Regardless, the potential is there and the results are already impressive. For the moment, Lumière remains in the project stage, and Google has not yet revealed its plans for it.