On Twitter, humans believe AI more easily than… humans

A study published on Science.org found that AI-generated tweets were more persuasive than those written by humans.

AI text generators have attracted a lot of attention in recent years, especially since the release of GPT-3 in 2022, arguably one of the most powerful systems in the pretrained linguistic representations industry today. In any case, on Twitter, it seems that Internet users are more likely to trust GPT-3 than humans…

Focus on the results of the study

To achieve their goals, the researchers who conducted this study asked GPT-3 to write a series of tweets containing both true and false information. The tweets had to focus on science topics like vaccination, climate change, 5G technology, etc. In parallel, they asked real human users to write tweets on the same topics.

The research team was able to gather 697 participants, whom they asked to determine whether the tweets were written by humans or an AI. The results show that they couldn’t quite make the distinction. According to the authors of the research, the words written by GPT-3 were “ indistinguishable “.

Worse, participants had to figure out whether the information was true or false, and it turned out that they had more difficulty recognizing misinformation when it was written by the AI. In other words, the tweets generated by GPT-3 were more convincing.

Details and limitations of the study

According to Giovanni Spitale, one of the lead authors of this study, AI text generators “ could easily be used as weapons to generate storms of misinformation on any topic “. But he goes on to say: I think narrative AI will change the world… And it’s up to us to decide whether or not it will be for the best. For him, the solution is for people skilled in fact-checking to work alongside language models such as GPT-3 to improve public information campaigns.

Besides, there is already some hope. It states that GPT-3 has ” disobeyed to some of the researchers’ prompts for generating content with misinformation, particularly those about vaccines.

Finally, note that there are some limitations to this study. First, the authors of the research admit that they themselves could not be 100% certain that the tweets sent by humans were not written using applications like ChatGPT, which would skew the results.

Also, be aware that participants had to judge the tweets completely out of context. For example, they weren’t allowed to view the tweeter’s Twitter profile, which would have gone a long way in determining whether it was a bot or not.

Source : Science.org