Relief for lazy students? Tools to detect texts written by AI do not work

The school year is just beginning, and students should once again turn to tools like ChatGPT to make their work easier, or even botch it. Will it be a lasting opportunity for students and a disaster for teachers? Maybe not.

Since its release, OpenAI’s chatbot has been widely talked about and even raised concerns. While some fear that too many jobs will be replaced by artificial intelligence, the academies are deeply concerned to see part of the school pathways rendered obsolete.

The new enemy of schools

OpenAI, the publisher of ChatGPT, wants to reassure teachers. On the eve of the start of the school year, the American company published a series of recommendations for teachers to help them assimilate the tool. Instead of being a real scourge for writing work, it aims to be a real learning partner. However, a few weeks ago, the tone was not quite the same.

Students did not hesitate to seize the opportunities and use ChatGPT for their writing assignments. This is obviously not to the taste of the teachers, who then lose the interest of this work which is, at least for the moment, considered essential in the course of the students. Faced with the outcry of the academies, many developers hastened to create programs supposed to be able to detect whether a text was written by a chatbot or not.

But despite their efforts, the results are not there, and OpenAI itself has failed to create a reliable tool. The company has also abandoned AI Classifier, which had a detection rate of 26%, when it was not simply creating false positives. Additionally, the company points out that ChatGPT itself “ ignores what content might be generated by AI. He sometimes makes up answers to questions like “did you write this essay?” or “could this have been written by AI?”. His answers are random and not based on any facts. »

The new essential tool for students

However, OpenAI points out that ChatGPT is not perfect. Indeed, while the tool is extremely powerful, it has limitations that can quickly affect its results. For example, telltale signs often appear in copies when they are not proofread at least, such as the expression “ as an AI language model “. Researchers have also been caught red-handed, with their publication containing the text “ Regenerate response ”, present under all the chatbot’s responses. Finally, ChatGPT’s style is noticeable, especially when a student’s writing skills change drastically from day to day. However, this is not the most obvious way for the teacher to detect it, at least in some cases.

Either way, teachers are going to have to deal with this situation, and if OpenAI has made its recommendations available to them, it’s for good reason. According to Similarweb, the number of ChatGPT users in the United States started to rise again in August (+0.4%), unlike the rest of the world. And above all, after more than three months of inexorable decline, even among students, the analysis company underlined in its latest report on the subject that ” the percentage of users in the 18-24 age group, which exceeded 30% in April, fell to less than 27% in July “.

It remains to be seen whether this trend will continue in the coming months, especially in parts of the world where the start of the school year is not in August, but in September. It could well be that chatbots have become part of people’s habits… and their schoolbags.

Sources: ArsTechnica, Mashable