No sooner have we discovered image creation AIs than the giants of the sector are already starting to create videos. Facebook presented its “Make-a-Video” system a few days ago, which makes it possible to create short clips from captions blown by Internet users, and it is now Google’s turn to embark on the adventure. The search giant has released a research paper detailing the progress the company is making in the industry.
Poor quality videos
The tool, dubbed Imagen Video, works similarly to Dall-E 2 or Midjourney. With a simple textual suggestion, the artificial intelligence is able to create a short video reflecting the situation described as best as possible. The result is a 24 frames per second video displaying a resolution of 1280 x 768 pixels. Even better, according to the team behind the development of Imagen, the machine is capable of “create high-definition video with high image fidelity, high temporal consistency, and deep understanding of language.” On the examples put online by Google, we can for example see an astronaut riding a horse on Mars, a panda driving a car or a teddy bear doing the dishes.
Just like the creations of Dall-E or other AIs of the genre, the videos created by Imagen are full of small flaws that show the progress that still remains to be made in the field. The first limitation is obviously technical. The AI doesn’t actually directly create a 24fps video in HD. First, the machine outputs a tiny 24 by 40 pixel video at 3 fps. This video is then passed through an upscaling algorithm that will try to improve the quality and add images to make the whole thing smoother. The result is a video in HD format, but with many artifacts and strange distortions that leave little doubt that the video was created by a machine.
AI powered by “problematic data”
Creating good quality realistic images or videos remains a challenge for all these AIs, since the more detailed the content, the easier it will be to see flaws. This is even more noticeable on video creation AIs that have less content to train with. Not to mention the computing power needed to analyze and create a video. But that’s not the only problem with Imagen Video. As the researchers behind the project explain, these algorithms “were trained with problematic data” which tend to reflect “social biases and stereotypes”.
Due to these issues, Google is currently limiting access to its video creation AI, also highlighting the risks such a tool could pose in broadcasting. “hateful or harmful content”. A debate that joins the one opened a few years ago about deepfakes and other manipulated videos.