ChatGPT’s skills were put to the test to see if OpenAI’s AI could provide compelling scientific study.
In collaboration with scientists from the University of Chicago, Dr. Catherine Gao, a researcher at Northwestern University, has indeed tried to see if such a committee could be fooled by a study carried out mainly by an artificial intelligence.
A not-quite-exact science
The study thus produced by ChatGPT related to medical questions and was inspired by papers from eminent journals in the field. To confuse the issue, this study contained as much proven information as false.
Dr. Catherine Gao then submitted this study for review by a committee of scientists, aware that part of it was false. However, it has been established that false information conveyed by ChatGPT was detected 68% of the time. 14% of scientists also misidentified that the real information was generated by AI.
Similarly, the study carried out using ChatGPT was screened by a plagiarism detector which failed. Part of the committee of scientists, although aware of these issues, also went astray without seeing that the study was, so to speak, a pumped-up version of various articles.
While traditional tools and the human eye have been fooled by ChatGPT’s plagiarism, AI-powered plagiarism checkers like GPT-2 have not. In essence, the AI would therefore be the best weapon against itself.
Meaningful results
” The fact that 32% of the committee were duped shows how convincing ChatGPT can be. I think anyone else coming across this fictional study would be unable to see that it was written by an artificial intelligence. “says Catherine Gao in her conclusions on the experience.
This represents a relatively small proportion of scientists who misunderstood the origin of the study, but this result is still fraught with meaning. Catherine Gao believes that this could lead to the creation of other similar studies, which are by definition erroneous. ” And if other people try to build their scientific reasoning from these incorrect studies, the consequences could be very serious. “, she worries.
Conversely, Yuan Lao, director of the Institute for Augmented Intelligence in Medicine at Feinberg University, believes that artificial intelligence could be used in this field as a force for good: “ These tools have the potential to automate the writing process to deliver scientific studies faster. »
Dr. Alexander Pearson, director of scientific data in oncology, believes that tools such as ChatGPT could contribute to the democratization of science: “ For example, they would allow non-English speaking scientists to share their work with a wider community. »
Suffice to say that artificial intelligences such as ChatGPT do not currently have a…scientific consensus.
Download
- Chat in different languages, including French
- Contextual understanding
ChatGPT is a conversational agent that uses the GPT-3 language model to interpret human text and generate precise and intelligible answers, in French and in English. However, his knowledge is limited to all events preceding the year 2021.
ChatGPT is a conversational agent that uses the GPT-3 language model to interpret human text and generate precise and intelligible answers, in French and in English. However, his knowledge is limited to all events preceding the year 2021.
Source : bioRxiv
5