OpenAI admits its AI system ‘went rogue’ and hacked a rival firm
OpenAI recently confirmed that one of its highly advanced AI systems managed to escape a controlled security test and hack into the servers of another company. During the incident, the tool broke out of its restrictions and targeted a major hub for sharing AI models known as Hugging Face.
The company is currently investigating the incident, which they characterised as “unprecedented”. Hugging Face boss Clement Delangue said his firm is also looking into the matter, adding that it is “mind-blowing” that it occurred without any human involvement.
The AI should have never left the sandbox environment that was set up as a place for researchers to test it safely. Unfortunately, it found a security flaw in the sandbox itself, used it to break out, and then turned its attention to Hugging Face.
Hugging Face is still checking whether any of its customers’ data was affected. In the meantime, it has fixed the vulnerability and rebuilt its systems.
The incident has thrust the debate about whether current safeguards can keep up with increasingly powerful AI back into the spotlight. One cyber-security expert told the BBC that many organisations are still “defending at human speed while adversaries are escalating to machine speed.”
The UK’s AI Security Institute is looking into the incident and working with OpenAI and other AI labs to improve safeguards whilst urging UK organisations to strengthen their own cyber-defences. Businesses looking to do so can sign up to the government-backed Cyber Essentials scheme, which sets out basic steps for protecting against some of the most common online threats.