The voice cloning service has noted abusive misuse of its functions, in particular to make racist remarks in the mouths of stars.
Emma Watson reading an excerpt from Mein Kampf? In a video relayed on Twitter, Internet users can see the English actress reading the book written by Adolf Hitler. It is actually a deepfake, these digital manipulations making it possible to realistically replace a face or a voice and make them say things created from scratch.
This computer trick was carried out using a voice cloning tool developed by ElevenLabs thanks to artificial intelligence. In a series of tweets, the start-up explains that it is seeing a “increasing number of misuses” of its software while audio tricks abound on the internet.
London-based ElevenLabs recently launched a trial version of its“ultra-realistic text-to-speech tools” after raising $2 million. Users of the anonymous 4Chan forum have shared messages generated by the software and mimicking the voices of celebrities, such as American podcast host Joe Rogan, to make them pronounce racist, sexist and homophobic texts. In one of these excerpts, a voice clone of ultra-conservative commentator Ben Shapiro threatens to rape New York elected official Alexandria Ocasio-Cortez. The voices of directors Quentin Tarantino and George Lucas were also hijacked.
HAVEALSO – ChatGPT: “When are we going to stop the commodification of our existence?”
Strengthen the identification process
“We would like to solve this problem by putting in place additional protective measures”has tweeted Monday ElevenLabs, which did not refer to specific incidents. The start-up proposes to strengthen the user identification process, to better verify the copyright of submitted samples and to carry out manual checks.
Read alsoBehind ChatGPT, the shadow of the giant Microsoft
It is one of many companies developing artificial intelligence software for the general public. These tools have aroused strong interest since the launch at the end of last year of the conversational robot ChatGPT by the Californian company OpenAI, capable of answering a variety of questions and of writing texts. But they also come with deepfake fears. Last year, a Europol report warned of this technology which, by facilitating identity theft, risked becoming a basic tool for organized crime.
HAVEALSO – Artificial Intelligence: What is ChatGPT?