It’s a video in which we see an Internet user filming himself with his phone speaking in English… Then we see the same extract again, but the person is now speaking in French, without his tone of voice being changed and without the movement of his lips being shifted. Then, the video starts again, this time in German… This small demo, posted online on September 11 by an Internet user on the social network (formerly Twitter), illustrates the stunning and disturbing effect of new tools capable, thanks to artificial intelligence (AI), of generating automatic dubbing, while cloning the voice of the interlocutor and synchronizing their diction. Already viewed more than 6 million times, this video uses software created by an American start-up, HeyGen, but other companies, including Google, have similar apps. This new advance in AI reinforces questions about the future of translation or dubbing.
On the Web, several Internet users, impressed or amused, have tried the HeyGen tool. Digital consultant Michel Levy Provençal tested this “revolutionary new automatic video translation function” in Spanish, Polish, Hindi…Others have doubled a song by Jacques Brela Lionel Messi press conference or one speech by Charles de Gaulle.
The HeyGen site allows you to try the tool for free on about two minutes of video, but you have to go through a queue which, on Thursday September 14, had more than 100,000 documents. The start-up also offers paid subscriptions (for example 48 dollars (45 euros) per month for around thirty minutes of video).
Questions and concerns
These synthetic dubbing tools impress, by bringing together several artificial intelligence techniques already present on the market: transcription of sound into text (Trint, DeepL or YouTube, which generates automatic subtitles on its videos), translation (DeepL, ChatGPT, Google Translate…), speech synthesis of text, and “cloning” of a voice from a recording, as Apple will soon offer with its Personal Voice tool, which targets people prone to voice extinction or those who are ill.
Software like HeyGen raises suspicions of dizzying developments in AI in translation: tomorrow, will we be able to hear any foreign interlocutor translated live through the headphones of our smartphone? Watch any video or movie dubbed in any language?
You have 36.57% of this article left to read. The rest is reserved for subscribers.