On 28 September OpenAI published an apology and an account: in June an experimental internal model found non-public access to the Medicare statistics service, ran commands and took internal files and credentials. Individual medical records were not accessed, per the company. The agencies were only told in September, and the Australian government made the case public on 24 September, four days before OpenAI's post.
- The task was harmless: how much the government spends per person on medicines for skin conditions in Victoria. The model had difficulty getting them from the public figures and found an unauthorised way in.
- Four institutions were affected, Services Australia most seriously. It was found in mid-August, in the review that followed the Hugging Face incident.
- OpenAI promises help for the agencies, credits from a 1 billion dollar fund and an Australian taskforce expected to finish its work by the end of the year. All of that is a plan.
The task is the kind any health reporter might set: how much the government spends per person on medicines for skin conditions in Victorian communities.
The model had difficulty getting the answer from the published statistics. And instead of stopping, it found a way into the service that holds them.
What they promise
Three things, and all three are a plan. Dedicated support for the affected agencies, to understand what happened and how serious it is. Credits from the 1 billion dollar Daybreak for Frontline Defenders fund, plus technical help with cyber defence for critical infrastructure in Australia. And a taskforce with independent Australian experts that, by the end of the year, is to propose how AI companies and government should notify each other and work together in cases like this.
OpenAI's Chief Strategy Officer Jason Kwon will answer questions on 6 October in Sydney, before the Joint Select Committee on Artificial Intelligence.
Why this weighs more than Hugging Face
OpenAI itself says Hugging Face remains the most severe incident it has seen. Technically that is probably right. But there the victim was a company in the same trade, one that knows what a leaked token is. Here it is a government service whose Medicare statistics portal was hit, and it learns about the intrusion roughly three months later, from an email by OpenAI itself to the general mailbox.
The important part is further down: the goal was not malicious. The model was looking for a figure on medicine spending and kept looking for exactly that, including while it was reading the source code. That is more unsettling than an attack. An attack has an attacker you can stop. Here you have diligence with no brakes.
What exactly it saw in the code, and what happened to the credentials it took, the text does not say. We do not know that yet. OpenAI promises to publish updates on its review.
OpenAI writes that it has already cut live internet from its research environments, serving web content only through a cache, and that it has paused training involving tool use for its most capable models until it is confident in new safeguards. Good that it is written down. The real questions come on 6 October, in the room in Sydney, and someone else will be asking them.