Researchers create first-ever computer worm capable of spreading through AI systems

You were wondering if artificial intelligence could be infected by viruses or malware ? Now yes.

This is, according to the team that developed it, a great first in the world of artificial intelligence. In order to remind the world of the risks inherent in any connected technology, a group of researchers has just created the very first computer worm capable of infecting AI agents. A demonstration which should alert to the possible emergence of new types of cyberattacks.

The resurgence of the Morris specter in the age of artificial intelligence

Named Morris II in reference to Morris, infamous for having wreaked havoc on the Internet in 1998, this new kind of computer worm would have the capacity to spread from one AI system to another. Created by a team of researchers at Cornell University (NY) in a test environment, Morris is now capable of attacking generative AI agents to siphon data and spread malware. As part of the tests carried out by the research group, the worm was able to attack an autonomous messaging assistant to steal information contained in emails and send spam.

An experience that highlights the growing vulnerability of large-scale language models (LLMs), as they become more and more efficient, versatile and multimodal. Although these AI worms have not yet been observed in real-world conditions, they represent a security risk that startups, developers and tech companies must take seriously and anticipate.

Self-replicating prompts to jailbreak generative AI systems

To create this worm, the Cornell Tech teams created a “self-replicating” prompt. In short, a set of instructions leading a generative AI system to self-produce additional instructions in its responses. A process found in classic SQL injection or buffer overflow attacks. At the same time, they also created an email system capable of sending and receiving messages using different intelligent agents such as ChatGPT and Gemini, as well as the open source LLM LLaVA.

The experiment was successful using two methods of self-replicating instructions. The first via text prompts, the second by integrating a prompt into an image file. In the first case, the researchers managed to “poison” the database of messaging assistants, forcing the triggering of RAG – retrieval-aumented generation, a technology allowing to improve the responses of the AI ​​using more recent information without modifying the LLM in depth.

Once the email is retrieved by the RAG in response to a user request, and sent to GPT-4 or Gemini, it manages to bypass the security measures of the AI ​​systems and steal the message data. “ The generated response containing sensitive user data later infects new hosts when used to respond to an email sent to a new client and then stored in the new client’s database “, explained Ben Nassi, one of the members of the research group.

With the second method (integration of the malicious prompt into an image), the process made it possible to force the assistant to transfer the infected email several times to other messaging systems.

To anticipate such attacks, which they believe are likely within two to three years, Cornell Tech researchers shared their work with Google and OpenAI who are working to make their systems more resilient, while encouraging developers to ensure that They work with healthy tools.

Source : Cornell Tech, Wired