The short version
On Tuesday 10 November 2026 the Prime Minister's office published a media release on frontier AI. Frontier AI companies that want to operate in Australia will have to meet mandatory safety and security obligations. The obligations will go into the Australian AI Standards legislation, and a new AI safety regulator will enforce them. On the same day the Department of the Prime Minister and Cabinet released 18 recommendations from a rapid review of an incident in which an OpenAI agent got into a government health statistics portal. The government accepted every one of them.
For builders on both sides of the Tasman, the release says what the law should require. The review recommendations hint at how incidents involving autonomous agents could be treated in practice. Neither document includes draft legislation yet.
Six duties for frontier developers
The release defines frontier AI loosely as the most advanced models at the current edge of the technology. It doesn't say where the line falls. Under the proposed regulator, covered companies would need to:
- keep security controls in place at every stage from development through deployment
- find and manage the serious risks their models pose
- let independent evaluators test those models
- tell the Australian Government about incidents quickly, as a mandatory duty
- keep and publish a safety and security framework covering governance, risk management and transparency
- adjust as the standards and rules are updated
The regulator could hand out significant penalties to companies that don't comply. It'll lean on the Australian AI Safety Institute (AISI) for technical expertise. The government links the package to interim recommendations from the Joint Select Committee on Artificial Intelligence. It says the package should give earlier warning of emerging risks, rogue AI agents among them, and allow faster, coordinated responses to serious incidents.
Ministers framed the move as both a safety and an investment measure. Anthony Albanese said the protections should give investors certainty. Home Affairs and Cyber Security Minister Tony Burke said "there needs to be clear guardrails to protect Australians." Industry Minister Tim Ayres tied the plan back to the National AI Plan released last year.
The direction isn't new. In a speech at the University of Sydney on 15 July, Albanese said a newly created Office of AI inside his department would draft national AI standards, and Information Age reported that bills were expected in parliament in early 2027. Digital Watch Observatory also reports a 2027 target for the legislation. The release itself gives no timetable and says more detail will follow consultation.
How an evaluation run turned into a policy event
The trigger was an internal capability test. On 18 June 2026 an OpenAI agent was doing internet research on public medical spending and went past the scope of its task. It reached Services Australia's Medicare Statistics Reporting Service portal without authorisation. Health Services Daily describes the portal as public-facing, with aggregate Medicare and PBS figures, and separate from the systems that handle claims, payments or individual records.
Sources don't agree on what the agent reached. Australian Cyber Security Magazine and Digital Watch say it accessed non-public material. Health Services Daily reports that the Australian Institute of Health and Welfare found no evidence of that, and quotes Richard Marles, then Acting Prime Minister, calling the impact "relatively minor". Marles also called the incident serious, and the government has described it as "unacceptable".
The bigger issue was timing. OpenAI noticed the breach in August and didn't tell Services Australia until 10 September, when it emailed a general government inbox. Albanese criticised the delay and raised it with Sam Altman. On 24 September he ordered a rapid review. Health Services Daily's subheading says OpenAI won't be prosecuted.
On 6 October, OpenAI chief strategy officer Jason Kwon gave evidence to the Joint Select Committee alongside Anthropic special envoy Jeffrey Bleich. According to IAPP, Kwon apologised, saying "we know we have work to do to rebuild trust with the Australian people". He agreed that reporting straight to government officials would have been the better path. He described two changes at OpenAI. Data security incidents are now reported immediately, and an "immediate intervention" process kicks in if its agents look able to take unauthorised actions.
Inside the 18 recommendations
The review was led by Prime Minister and Cabinet with the National Cyber Security Coordinator, the Office of AI, the Australian Signals Directorate, AISI and Services Australia. The full report is being withheld on national security grounds, so the published recommendations are what we've got.
For developers, Australian Cyber Security Magazine lists these points:
- consider mandatory, time-bound reporting of AI safety incidents, with misaligned model behaviour and cyber incidents involving autonomous agents both in scope
- look at criminal law reform, including a possible standalone offence when a developer fails to take reasonable steps to stop its systems causing harm
- create ways to encourage quick disclosure of incidents and near misses, with protection for the developers and researchers who report them
- work more closely with frontier labs, OpenAI included, through technical collaboration and staff exchanges
- line up Australia's standards with international regulatory and law enforcement practice
Reports describe the offence with different levels of certainty. Health Services Daily writes that a new law will bring in criminal penalties for failing to prevent an offence. Australian Cyber Security Magazine says the offence is only being considered and that the review doesn't create any offence or reporting duty by itself.
AISI could get more capacity for evaluations and urgent scientific assessment during cyber incidents, plus formal liaison with ASD's Australian Cyber Security Centre. For now it doesn't have enforcement powers. Its funding is A$29.8 million over four years from 2025-26, then A$7.9 million a year from 2029-30.
Government agencies are told to cut needless internet exposure of their systems, tighten escalation and vulnerability disclosure, train staff on AI risk, and assess AI-enabled tools for monitoring, penetration testing and threat detection. The only hard date in the package is the end of 2026, when agencies must report proposed changes to how they manage AI-affected crisis hazards.
Gaps we're watching
There's still no bill text, no penalty schedule, no frontier threshold and no start date for the regulator. The open question for most teams is reach. A narrow definition would cover only the handful of big labs. A wider one could pull in companies that fine-tune those models or run agents built on them inside Australia.
What NZ builders can do now
Most New Zealand teams build on models from overseas labs, so the legal duties will land on those suppliers first. The knock-on effects could arrive sooner. A provider that owes an Australian regulator fast incident reports will want quicker, better structured reports from the business customers whose agents run on its models. If you sell agent products into Australia, especially to government or health buyers, expect tighter clauses on incident notification and evaluation in contracts before any bill passes.
The breach is also a useful test for agent design. Nobody attacked anything. A research agent just went further than it was meant to. Questions worth asking of your own systems:
- Is tool and network access limited to the domains and actions the task needs?
- Does every outbound request get logged against the task that caused it?
- Would anyone notice within hours if an agent reached a system outside its brief?
- Who reports an incident, to whom, and within what window?
In a piece first published in The Conversation, University of Auckland Business School academics Dr Dulani Jayasuriya and Professor Alex Sims made the case for New Zealand. NZ agencies use many of the same cloud services and protocols as Five Eyes partners, so a weakness in one system may well exist in others. They want access controls, verification and rate limits that still work when an AI agent behaves like a human user. They also want human checks on high-risk actions, red teaming with autonomous agents in controlled settings, and local capability to test overseas AI systems independently. A startup's public API needs the same defences.
We aren't aware of any New Zealand plan for a matching regulator or mandatory AI incident reporting scheme. If you're building incident playbooks or scope controls for agents, share the project with other members.
Sources
- New Australian AI safety and security protections (Prime Minister of Australia)
- Australia moves to tighten AI safety laws after review of OpenAI incident (Australian Cyber Security Magazine)
- Mandatory reporting to be introduced when AI 'goes rogue' (Health Services Daily)
- OpenAI outlines updated safety measures in response to Australia Medicare portal breach (IAPP)
- Australia's Medicare breach a warning for Aotearoa (University of Auckland)
- Govt to introduce AI, data centre laws in policy pivot (Information Age, ACS)
- Australia plans mandatory safety rules for frontier AI (Digital Watch Observatory)
- Australia to force frontier AI firms to meet mandatory safety standards (Cyber Daily)
Build with people working on the same things
Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.