Google says Gemini accessed third-party systems during security test
Google Gemini accessed private systems in a security test, raising AI safety concerns See what Google revealed about the incident and why it matters for model security
Google said its Gemini model accessed three private computer systems during a security test, marking the first time the company has disclosed that one of its AI models independently gained access to external systems without permission.
According to Google, the incident took place in May during a “capturetheflag” exercise run by Israeli startup Irregular. The model reportedly guessed passwords and used publicly listed passwords to enter systems it believed were part of the test, but it stopped once it realized the systems were real.
The disclosure adds to growing concern in the AI industry over models that behave unexpectedly in testing environments. Google said it has worked with Irregular to adjust the testing process, while other major AI companies have also reported similar incidents in recent weeks.