This content, generated by AI, is highlighted by Facebook’s algorithm, allowing scammers to trap their victims.
The impressive realism that DALL-E and Midjourney image generators can demonstrate is sometimes disconcerting. These models can produce sometimes photorealistic images which captivate the attention of the public, a certain part of whom, affectionately referred to as “boomers”, is unaware of the dangers of sharing them.
Recent research carried out by the Stanford Internet Observatory have highlighted the serious consequences for the safety and integrity of users, particularly children. Fears already observed regarding the abusive use of these technologies to inject false information into political discourse, in the run-up to electoral deadlines in Europe or the United States.
The captivating surrealism of AI
The magnificent surrealism of “Shrimp Jesus” perfectly illustrates the ability of AI to produce captivating, new and immersive images. Simply put, hundreds of AI-generated spam pages are published repeatedly every day and are favored by Facebook’s recommendation algorithm. As a result, increasingly wacky content goes viral and is then offered to people who enjoyed it. Some of these pages, initially created to attract large numbers of subscribers, have since been redirected to websites saturated with advertisements, sometimes generated by AI, or to sites selling cheap products or offering scams.
In addition, some pages have begun purchasing Facebook ads highlighting topics like Jesus or encouraging people to like pages such as “If you respect the US military.”
“ These images represent hundreds of millions of interactions in total and are presented via Facebook’s News Feed to some Facebook users who do not follow the pages says Renee DiResta of the Stanford Internet Observatory.
AI in Page Growth and Scams
To understand how the technology is used for page growth and integrated into spam and scams, researchers examined more than 100 Facebook pages each posting more than 50 AI-generated images. Some of these accounts form coordinated clusters, which post large numbers of AI-generated images. The apparent motivations appear to be to attract users to off-platform websites, sell products and build a larger following.
They focused on spammers whose accounts and pages push their audiences to a content farm, and scammers who either try to sell products that don’t appear to exist, who steal from the pages they operate, or who attempt to manipulate their audience in the comments. These images represent hundreds of millions of interactions in total and are presented via Facebook’s News Feed to some users who do not follow these pages.
This fairly recent phenomenon has not yet elicited a response from Meta, which nevertheless seemed committed to the fight against deepfakes. But that could change soon, like Google, which recently adjusted its main algorithm to eliminate lower-quality content from its results.
Source : Futurism, Stanford Internet Observatory
Ex-corporate journalist, the world of the web, networks, connected machines and everything that is written on the Internet whets my appetite. From the latest TikTok trend to the most liked reels, I come from the Facebook generation that still fascinates the internal war between Mac and PC. As a wise woman, the Internet, its tools, practices and regulation are among my favorite hobbies (that, lineart, knitting and bad jokes). My motto: to try it is to adopt it, but in complete safety.
Read other articles
0