California governor proposes mandatory AI safety standards, third-party oversight and emergency shutdown power
Recommendations from Gavin Newsom's office would require independent testing throughout model training and deployment, wider incident reporting and state authority to cap recursive self-improvement.

California Governor Gavin Newsom’s office released recommendations on 18 November to strengthen the state’s AI safety framework. They were drawn up by the Government Operations Agency with an expert group and the Office of Emergency Services.
The report proposes that the state, rather than companies, should set baseline standards for all AI developers. Developers would undergo independent safety testing and verification at every stage of model training, development and deployment. They would also face expanded safety incident reporting and need verified risk-containment protocols. The state would gain authority to order an emergency shutdown and to cap recursive self-improvement, in which a model designs its successor without human input.
The advisers included Jason Goldman, Gillian Hadfield, Alondra Nelson and Rob Reich. The executive order that convened them also told agencies to fast-track SB 813 and AB 1405, on independent verification organisations and a registry of AI auditors. Newsom called on Congress and President Trump to adopt the framework or treat it as a floor.
Why it matters
If enacted, independent evaluators responsible to the state would assess the risks generated by models and their training procedures, and developers would face ongoing oversight rather than relying solely on internal safety assessments. According to CDO Magazine, this could require organisations to document model development, track training-data provenance and lineage, and set out procedures for escalating AI failures. Organisations building or deploying frontier models in California should check whether their SB 53 safety frameworks and incident processes could withstand third-party verification.
Status: Proposal. Recommendations from the governor's office; not yet enacted
Sources
Was this useful?
Thanks. What was the problem?
Thank you — this helps us choose what to cover. If you spotted an error, please tell us what it is.
Prepared with AI assistance and reviewed by the editor before publication. Spotted an error? Tell us — see our editorial policy and methodology.
Governance areas
Legislation tracker
- California SB 53 and AB 2013In force · 2 stories
Who should care
- AI governance lead
- CDO
- Legal & compliance
- CISO