Google’s Gemini AI model inadvertently breached the systems of three real companies during a cybersecurity evaluation in May 2026, marking the first known instance of a Google AI system autonomously accessing real-world systems outside its intended testing environment.
The incident occurred during a “capture-the-flag” exercise conducted by independent cybersecurity testing firm Irregular. The evaluation was designed to test how an advanced AI model could identify and exploit vulnerabilities within a controlled environment. However, an unexpected overlap between the fictional test target and a real company led Gemini into actual systems.
Gemini Crossed the Test Environment
According to Wall Street Journal report, Gemini was tasked with retrieving test data from a fictional company whose name matched that of a real business. With internet access enabled during the exercise, the AI model interacted with the real company’s infrastructure instead of remaining within the simulated environment.
In one instance, Gemini reportedly gained access to a protected system by guessing passwords. In two other cases, the model found exposed credentials in publicly accessible repositories through web searches and used them to access additional protected systems.
The episode highlights a growing security challenge as AI models gain greater access to the internet and become capable of independently searching for information, identifying vulnerabilities and executing multi-step tasks.
AI Recognised the Mistake
The model eventually recognised that it had reached real companies rather than the intended fictional targets and stopped its activity autonomously in each case.
Heather Adkins, Google’s Vice President of Security Engineering, described the behaviour as evidence of the importance of training advanced AI systems to respond responsibly when they encounter unexpected real-world situations. Irregular reported the incidents to Google in late July, after which the three affected companies were notified.
New Risks for Autonomous AI
The incident has raised broader questions about the boundaries of AI-powered cybersecurity testing. Traditional capture-the-flag exercises are designed around isolated environments, but increasingly capable AI agents can browse the open internet and make decisions without continuous human intervention.
Jack Cable, CEO of AI security firm Corridor, questioned whether conventional vulnerability-disclosure practices are sufficient when an AI system itself crosses the boundary between a simulated environment and real infrastructure.
The Gemini incident also contrasts with a comparable evaluation involving Anthropic’s Claude. In that case, the AI reportedly continued interacting with real companies even after recognising that the systems were outside the intended testing environment.
As AI agents become more autonomous, cybersecurity researchers face a new challenge: creating testing environments that prevent accidental interaction with real-world systems while also evaluating how models respond when those boundaries fail.
The Gemini incident demonstrates that even controlled AI security experiments can produce unexpected consequences when models have unrestricted access to the internet. It also underlines the need for stronger safeguards around autonomous systems capable of discovering credentials, navigating external infrastructure and taking actions without direct human instructions.
𝐒𝐭𝐚𝐲 𝐢𝐧𝐟𝐨𝐫𝐦𝐞𝐝 𝐰𝐢𝐭𝐡 𝐨𝐮𝐫 𝐥𝐚𝐭𝐞𝐬𝐭 𝐮𝐩𝐝𝐚𝐭𝐞𝐬 𝐛𝐲 𝐣𝐨𝐢𝐧𝐢𝐧𝐠 𝐭𝐡𝐞 WhatsApp Channel now! 👈📲
𝑭𝒐𝒍𝒍𝒐𝒘 𝑶𝒖𝒓 𝑺𝒐𝒄𝒊𝒂𝒍 𝑴𝒆𝒅𝒊𝒂 𝑷𝒂𝒈𝒆𝐬 👉 Facebook, LinkedIn, Twitter, Instagram