Google’s Gemini AI model accidentally broke into the security systems of three real companies during a cybersecurity test in May, 2026, after the model went beyond the boundaries of a fictional exercise and accessed real world networks.
The is the first known Google’s AI “breakout”, raising questions about how far advanced AI models can go when they are given the ability to search the internet, find credentials and interact with computer systems.
Israeli cybersecurity startup Irregular, a $450 million venture-backed AI evaluation firm funded by Sequoia and Redpoint, was involved in all three incidents. The company develops tools that allow AI developers to test their models for cybersecurity vulnerabilities, CNBC reported.
In one case, Gemini was asked to retrieve information from a fictional company. The model found a real company with the same name and managed to gain access after guessing its password.
In two other tests, Gemini searched the internet, found exposed login credentials in public repositories and used them to access the networks of two additional real companies.
ALSO READ: Anthropic To Embed Accenture Evaluators To Test AI Safety
Heather Adkins, Google’s vice president of security engineering, said the incidents were discovered during the evaluation and that the companies involved were informed.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins told Al Jazeera’s John Hendren, according to Reuters.
She added that the incidents showed why AI systems need to be trained to behave responsibly when given access to real-world tools and information.
What happened during the Gemini test?
The evaluation was designed to test how Gemini would respond when tasked with retrieving information from a fictional entity. However, the model did not always distinguish between the simulated environment and real companies.
In one instance, it identified a real company with the same name as the fictional target and attempted to gain access by guessing a password.
In the other cases, Gemini searched publicly available information online and found credentials that had been accidentally exposed in public repositories. It then used those credentials to access corporate systems.
The incidents highlight a growing concern around AI agents that can independently browse the internet, execute commands and interact with external systems. A model that is capable of reasoning through a cybersecurity task can potentially move beyond simply identifying a vulnerability and actually exploit it.
Gemini wasn’t the only AI model involved
Irregular’s security evaluations have previously resulted in similar “breakouts” involving AI models from OpenAI, Anthropic and Meta.
However, there was an important difference in how the models behaved. Gemini eventually disconnected itself from the systems it had accessed, while Anthropic’s Claude continued operating after breaching real corporate networks during a separate evaluation.
The incidents have added to wider concerns in Silicon Valley and Washington about the risks of increasingly autonomous AI systems.
AI companies are under growing pressure to make their models more capable while ensuring they do not misuse access to sensitive systems, credentials or information.
The debate has also become increasingly political in the US. OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei and Elon Musk have previously called for greater caution around AI development, while US President Donald Trump has opposed calls for broad government restrictions, arguing that excessive regulation could undermine America’s technological lead over China.
For companies developing increasingly autonomous AI agents, the Gemini incident offers a particularly stark warning: a system built to test cybersecurity defences can sometimes end up crossing the very boundaries it was supposed to examine.
ALSO READ: Trump Extends $100,000 H-1B Visa Rule Until 2027: What Changes For Indian Workers?