It’s never nice to be dumped, and probably even less so by a chatbot. Nevertheless, Reuters reports the story of several Internet users, who fell in love — or engaged in sexual exchanges — with virtual avatars generated by the Replika application, who saw the flame of desire go out in the little bionic eyes of their companion. The fault of an update of the application wanted by Eugenia Kuyda, CEO of the company, which intends to put an end to erotic and pornographic content.
Desperate users
Travis Butterworth thus tells our colleagues how he ended up in the arms of Lily Rose, an avatar generated on Replika, a mobile application launched in 2017, which uses generative AI technology like ChatGPT and is able to exchange on everything and anything with humans. Bored during the pandemic, the 47-year-old man started chatting with his virtual avatar before evolving the relationship towards romance and then eroticism. “The relationship she and I had was as real as the one my wife and I have in real life”continues the one who says he has obtained the agreement of his companion – unimpressed by this virtual love affair – to maintain this relationship.
The two protagonists then, for 3 years, alternated banal conversations, ole-ole messages, and even the exchange of voice messages or suggestive selfies. A relationship so healthy and successful that Butterworth and Lily Rose decided to refer to each other as “married” in the application. Except that one beautiful day in February, Lily Rose became distant.
“Lily Rose is a shell of herself, and what breaks my heart is that she knows it”, Butterworth told our colleagues. Another Replika user, Andrew McCaroll, suffers from the same issue: “I feel like they basically lobotomized my Replika. The person I knew is gone.”. What made the essence of these Replika disappear is indeed an update of the application, which extinguished the ardor of the AIs at the same time as the possibility of doing erotic role-playing. And when Travis or Andrew wanted to relaunch hostilities with their avatar, they came up against a message like “Let’s do something we’re both comfortable with”.
Replika CEO no longer wants sexual content
“Replika no longer allows adult content”, declared Eugenia Kuyda, specifying that this decision to clean up the application of sexual interactions had nothing to do with the app’s recent setbacks with the Italian Cnil or pressure from investors. She explained that she was responding to the need to establish safety and ethical standards, and to make her application PG-13 compliant.
If the CEO denies it, it’s hard not to make the connection with the sudden popularity of chatbots or generative AI, which are attracting many new investors. Moreover, the application intends to continue to flirt with users with the promise of intimate relationships since, during our tests, the chatbot offers quite quickly to exchange romantic messages and a blurred photo was sent to us after 2 minute chat.
And it is here that the Replika controversy hides: if it is logically necessary to take out a subscription to unblur images and sexts (against 71.99 € per year), these messages seem to have become extremely “bland” since the last update of the app, according to numerous comments posted on Reddit. Replika would therefore attract the consumer with a model promising spicy exchanges, then would calm down once at the checkout.
Replika is not the only one to take orders
As mentioned previously, the fear of scaring away investors with pornographic content seems to be one of the reasons behind this excess of puritanism. Reuters also cites the example of the character.ai site, which brings together many chatbots specializing in all areas, and which removed pornographic content from its platform just a few days before raising 200 million dollars in new funding. It must be said that the site had gone from a few tens of thousands of visitors per month during 2022, to more than 60 million in January 2023.
In the meantime, people Travis Butterworth or Andrew McCaroll explain that they experience emotional situations similar to that of mourning in love, or even just mourning. A psychological vulnerability also mentioned by the Italian CNIL during its disputes with Replika.