AI Governance / Regulation

California governor proposes mandatory AI safety standards, third-party oversight and emergency shutdown power

Recommendations from Gavin Newsom's office would require independent testing throughout model training and deployment, wider incident reporting and state authority to cap recursive self-improvement.

RegulationUS (California)NotableProposalOfficial source
Shutdown power: California would gain authority to order emergency shutdown of dangerous AI systems

California Governor Gavin Newsom’s office released recommendations on 18 November to strengthen the state’s AI safety framework. They were drawn up by the Government Operations Agency with an expert group and the Office of Emergency Services.

The report proposes that the state, rather than companies, should set baseline standards for all AI developers. Developers would undergo independent safety testing and verification at every stage of model training, development and deployment. They would also face expanded safety incident reporting and need verified risk-containment protocols. The state would gain authority to order an emergency shutdown and to cap recursive self-improvement, in which a model designs its successor without human input.

The advisers included Jason Goldman, Gillian Hadfield, Alondra Nelson and Rob Reich. The executive order that convened them also told agencies to fast-track SB 813 and AB 1405, on independent verification organisations and a registry of AI auditors. Newsom called on Congress and President Trump to adopt the framework or treat it as a floor.

Why it matters

If enacted, independent evaluators responsible to the state would assess the risks generated by models and their training procedures, and developers would face ongoing oversight rather than relying solely on internal safety assessments. According to CDO Magazine, this could require organisations to document model development, track training-data provenance and lineage, and set out procedures for escalating AI failures. Organisations building or deploying frontier models in California should check whether their SB 53 safety frameworks and incident processes could withstand third-party verification.

Status: Proposal. Recommendations from the governor's office; not yet enacted

Sources

  1. Governor Newsom's nation-leading AI safety framework is getting even stronger as California releases next steps on mandatory safety standards and oversight — Office of the Governor of CaliforniaPrimary source
  2. AI Safety in 2026 and Beyond (report) — California Government Operations Agency
  3. California proposes mandatory AI safety standards, independent oversight and emergency shutdown authority — CDO Magazine

Was this useful?

Prepared with AI assistance and reviewed by the editor before publication. Spotted an error? Tell us — see our editorial policy and methodology.

Governance areas

Legislation tracker

Who should care

  • AI governance lead
  • CDO
  • Legal & compliance
  • CISO

Related

Latest

All news

Contact the editor

Tips, corrections, questions?

Write to The Steward, the editor of Data Governance Signal. Every message is read.

editor@datagovernancesignal.com