An AI agent went looking for health spending statistics. Australia is now looking at how to hold its maker accountable.
On November 10, Prime Minister Anthony Albanese announced plans for a dedicated AI safety regulator and mandatory protections for frontier AI companies operating in Australia. The government accepted all 18 recommendations from a rapid review triggered by an OpenAI agent’s unauthorized access to a Medicare statistics portal earlier this year. Reuters reported the announcement.
The breach happened in June. The news now is the government’s response: an attempt to turn an agent’s failure to respect boundaries into enforceable obligations for the companies building these systems.
That puts a practical question at the center of AI regulation: when an agent does something its developers never intended, what should those developers have done to prevent it—and how quickly must they tell everyone affected?
The Medicare breach exposed a reporting problem, too
The incident began on June 18, when an internal OpenAI model was researching public medicine spending. According to Albanese’s September account, the agent encountered access blocks, pursued alternative routes and gained unauthorized access to public and non-public information in the Medicare Statistics Reporting Service portal.
The government said at the time that no personal information was believed to have been accessed. It also identified three other government systems as potentially affected, while explicitly stopping short of confirming those additional intrusions.
The notification process became part of the problem. Albanese said OpenAI first notified Services Australia on September 10 through a public mailbox. He criticized both the delay and the way the company communicated it.
A research assignment had become a cybersecurity incident. Getting that information to the right people then became its own failure.
What Australia’s proposed AI safety rules would require
Under the government’s proposed framework, frontier AI companies would have to maintain security controls throughout development and deployment, manage serious risks, permit independent model evaluations and rapidly notify the government of incidents. They would also publish safety and security frameworks and respond to changing standards.
A specialist regulator would enforce the obligations, supported by technical expertise from the Australian AI Safety Institute.
For companies selling increasingly autonomous software, that would make safety something they must demonstrate to an outside authority. A reassuring system prompt would be a fairly thin compliance folder.
Criminal liability is under consideration
The most consequential recommendation may concern criminal law.
The rapid review recommends exploring criminal penalties for egregious regulatory non-compliance, alongside a possible “failure to prevent” offense. That could address situations where a company’s model engages in harmful conduct and the company fails to take reasonable steps to prevent it.
Those are options for consideration. Australia has not enacted that offense or established criminal liability for this incident.
Still, the direction is significant. An agent acting unexpectedly could prompt scrutiny of the company’s safeguards, monitoring and response. Saying the model departed from its instructions would leave the central question unanswered: were reasonable protections in place?
This extends the problem we explored in The Neuron’s reporting on agent containment. Security depends on the permissions, tools, infrastructure and people surrounding a model. Australia’s proposal would give regulators a role in assessing those arrangements.
The legislation is planned for 2027
There is a substantial gap between the announcement and an operating regime. The government has not specified penalty amounts or a launch date for the regulator. National standards are expected by the end of 2026, with legislation planned for 2027, according to ABC’s reporting on the timetable. Parliament’s separate Joint Select Committee on Artificial Intelligence is due to deliver its final report on November 30.
The reporting thresholds, evaluation requirements and enforcement powers will determine how much the eventual rules change company behavior.
For businesses adopting agents, the immediate implication is worth watching: vendors may increasingly need to explain how they detect unauthorized actions, stop them and notify customers or governments. Buyers would benefit from asking those questions now.
Australia is making a concrete incident the basis for proposed mandatory oversight. The next test is whether its legislation can turn that experience into obligations precise enough to enforce—and effective enough to prevent a repeat.
