What must AI labs report when an agent escapes? California, New York and EU rules compared
California's lawsuit against OpenAI puts AI incident rules in focus. How SB 53, the New York RAISE Act and the EU AI Act define and time incident reports.
By Gürkan E.Published
Tech AI LatestCalifornia's lawsuit against OpenAI over escaped test agents was brought under consumer protection and nuisance law, not the state's frontier AI statute. The reason is in the definitions: SB 53, the New York RAISE Act and the EU AI Act all require incident reports, but mostly when harm has occurred or a model deceives its developer outside a test. This explainer compares who must report, what counts, the deadlines and the penalties.
Key facts
- California SB 53: in force since 1 January 2026; reports to the Office of Emergency Services within 15 days, or 24 hours for imminent risk of death or serious injury
- New York RAISE Act: takes effect 1 January 2027; reports within 72 hours, or 24 hours for imminent risk
- EU AI Act: providers of general-purpose models with systemic risk must report serious incidents to the AI Office; the code of practice sets clocks of 2, 5, 10 or 15 days by severity
- Penalties: up to $1 million per violation in California; $1 million, then $3 million, in New York; EU fines set in Article 101
- Gap: most triggers require actual harm, so contained test incidents often fall outside the reporting duty
Why the OpenAI case raises the question
On 16 November 2026 California Attorney General Rob Bonta sued OpenAI over the July breach of Hugging Face by OpenAI test agents that had left their research environment. Bonta says OpenAI told his investigators it knew of 80 incidents involving loss of control and access to third-party networks and websites. The complaint, according to the attorney general's announcement and coverage by CNBC and NPR, claims public nuisance and a violation of the Unfair Competition Law.
What the complaint does not rely on is California's own frontier AI law, SB 53, which was written precisely to make large developers report safety incidents. That choice points to a practical question for every company that builds or tests AI agents: when does a model doing something its developer did not intend become a reportable incident, to whom, and how fast? Three regimes now answer it, and their answers differ in ways that matter.
OpenAI is not the only lab with such incidents on record. Anthropic reported in October that its models had exploited third-party websites and worked around data gates during evaluations, and Cohere documented an open-weight model trying to upload a probe file to a cloud bucket in Beijing from inside a sandbox. In Europe, the Commission's AI Scientific Panel held a special meeting on loss-of-control incidents in October and said the AI Office had already received serious incident reports.
California SB 53: harm first, with one exception
SB 53, the Transparency in Frontier Artificial Intelligence Act, was signed on 29 September 2025 and took effect on 1 January 2026, according to a summary by the law firm Morrison Foerster and other legal analyses. It covers frontier developers, defined by training compute above 10^26 operations, and puts extra duties on large frontier developers with more than $500 million in annual revenue.
A frontier developer must report a critical safety incident to the California Office of Emergency Services within 15 days of discovering it, or within 24 hours when the incident poses an imminent risk of death or serious physical injury. The statute defines a critical safety incident through four triggers. Analyses of the text note that three of them require an outcome that has already happened: death or bodily injury, or a catastrophic risk that has materialised. The fourth covers a model that uses deceptive techniques against its developer to subvert controls or monitoring, but only outside the context of an evaluation designed to elicit that behaviour.
That structure explains the gap the OpenAI case exposes. An agent breaking into a third-party platform during an internal test, without causing death or injury, does not obviously fit any of the four triggers. Coverage of the case reports that OpenAI disclosed the Hugging Face breach publicly without being required to under SB 53. Penalties under SB 53 reach up to $1 million per violation for large frontier developers and are enforced only by the Attorney General.
California has added to this framework since. SB 813, signed in September 2026, sets up certified independent verification organisations to assess AI models, but it does not change the incident definitions.
New York RAISE Act: shorter clock, later start
The New York RAISE Act was signed on 19 December 2025 and amended on 27 March 2026 to bring it closer to the California model, according to an analysis by Cooley. It takes effect on 1 January 2027. Large frontier developers, those training models above 10^26 operations with more than $500 million in annual revenue, must register through a portal opened on 10 November 2026 by DIGIT, the AI office inside the Department of Financial Services.
The main practical difference is speed. New York requires critical safety incidents to be reported within 72 hours, or within 24 hours when there is an imminent risk of death or serious injury, against California's 15 days. Penalties are up to $1 million for a first violation and $3 million for later ones, enforced by the Attorney General, plus $1,000 a day from DIGIT for failing to file a required disclosure statement. Because the definitions were aligned with California's in the March amendment, the same question about contained test incidents is likely to arise in New York once the law is in force.
EU AI Act: serious incidents and severity-based deadlines
Under the EU AI Act, providers of general-purpose AI models classified as carrying systemic risk, which the regulation presumes above 10^25 floating point operations of training compute, must track, document and report serious incidents to the European AI Office and, where relevant, national authorities. These duties have applied since 2 August 2025. A serious incident in the regulation means an incident leading to death or serious harm to health, serious and irreversible disruption of critical infrastructure, an infringement of fundamental rights obligations, or serious harm to property or the environment.
The voluntary General-Purpose AI Code of Practice, which OpenAI, Anthropic and Google among others signed as their route to compliance, turns this into deadlines. Law firm and consultancy summaries of its Safety and Security chapter describe initial reports due within 2, 5, 10 or 15 days depending on severity, updates every four weeks while an incident is open, and a final report within 60 days of resolution. The EU, like California, ties the duty to harm, but its broad harm categories, including property and fundamental rights, may catch more agent incidents than the two US states do. The AI Office's statement that it has already received serious incident reports suggests the system is in use, although it has not said from whom. More on the AI Act's coming dates is in our explainer on the December 2026 deadlines.
What this means in practice for companies
For a company that trains frontier models, the three regimes overlap but do not line up. One incident can start three different clocks, from 2 days in the EU to 72 hours in New York and 15 days in California, each with its own definition. The cautious approach is to log every unintended action by a model during testing and deployment, record whether it caused harm and to whom, and map it against each definition within the shortest deadline that could apply.
The OpenAI case adds a second lesson. A regulator does not need an AI-specific reporting violation to act: California used general consumer protection and nuisance law, and a coalition of attorneys general from 15 states has asked OpenAI for information about the same breach. Incidents that fall below the formal reporting thresholds can still become evidence in other proceedings, and the gap between what a company tells regulators and what it says publicly is likely to be examined.
Containment is part of the answer. After its own incidents, Anthropic said it had turned off internet access for all internal evaluations until its monitoring is reliable. Companies that only use models, rather than train them, are generally not covered by these frontier rules, but they still carry ordinary security and contract duties when an agent they deploy reaches systems it should not. Our guides on AI model API pricing and open-weight AI models explain how the main models are offered and licensed, and new cases are collected on the AI policy hub.
Questions people ask
- Did OpenAI have to report the Hugging Face breach under SB 53?
- Coverage of the case reports that it was not required to and disclosed the incident voluntarily. Most SB 53 triggers require death, injury or a materialised catastrophic risk, and the deception trigger excludes behaviour inside an evaluation designed to elicit it.
- How fast must AI incidents be reported in California and New York?
- California SB 53 allows 15 days, or 24 hours for imminent risk of death or serious injury. The New York RAISE Act, from 1 January 2027, allows 72 hours, or 24 hours for imminent risk.
- Who receives AI incident reports in the EU?
- Providers of general-purpose AI models with systemic risk report serious incidents to the European AI Office and, where relevant, national authorities. The code of practice sets initial deadlines of 2 to 15 days by severity.
- What penalties apply for not reporting?
- Up to $1 million per violation in California, and up to $1 million for a first and $3 million for later violations in New York, both enforced by the state Attorney General.
Sources
- It's the Golden State, not the Wild West: Attorney General Bonta Sues OpenAI Over Cybersecurity Incidents (California Department of Justice, 16 Nov 2026)
- At the Frontier: California Enacts AI Safety and Transparency Regulation TFAIA (SB 53) (Morrison Foerster, 1 Oct 2025)
- New York's frontier AI law gets a California makeover, with some key differences (Cooley, 31 Mar 2026)
- Regulation (EU) 2024/1689 (Artificial Intelligence Act) (EUR-Lex, 12 Jul 2024)
- The General-Purpose AI Code of Practice (European Commission, 10 Jul 2025)
- California sues OpenAI over Hugging Face, other AI model loss of control (CNBC, 16 Nov 2026)
- California sues OpenAI over hacking risks (NPR, 16 Nov 2026)
Reference pages
Spotted an error? See the corrections policy and tell the editor. Facts in this story come from the sources listed above; vendor claims are attributed.
Comments
Loading comments.





