Executive Summary
California Senate Bill 574 would establish the first statutory framework in the nation governing attorney use of artificial intelligence, transforming existing advisory ethics guidance into binding legal requirements.
- Confidentiality Protection: Prohibits attorneys from entering confidential, personally identifying, or nonpublic client information into public AI systems
- Verification Mandate: Requires attorneys to take reasonable steps to verify AI-generated content accuracy and correct hallucinated or erroneous outputs
- Personal Citation Review: Mandates that attorneys personally read and verify every citation in court filings, including those provided by AI
The proliferation of generative artificial intelligence tools in legal practice has created both opportunity and peril for attorneys. As AI-powered research assistants and drafting tools become ubiquitous in law firms across the country, California lawmakers are moving to establish clear statutory guardrails for their use. Senate Bill 574, currently pending in the State Assembly after passing the California Senate, would formalize the application of existing ethical guidance into specific binding legal requirements for AI use, with particular emphasis on protecting client confidentiality, ensuring citation accuracy, and maintaining human oversight of legal judgment.
California’s SB 574 is groundbreaking because it would transform the state’s advisory ethics guidelines into enforceable statutory requirements with specific legislative language. The states that have addressed AI ethics for attorneys (approximately 15+ states) rely on advisory ethics opinions interpreting existing Rules of Professional Conduct, while California is attempting to create new statutory obligations specific to AI use by attorneys. Many states have not yet issued any formal guidance on this issue.
The Hallucination Problem
The urgency behind SB 574 stems from a troubling pattern of AI-generated fabrications infiltrating court filings. For example, in October 2025, California’s Fourth District Court of Appeal imposed a $1,500 sanction on criminal defense attorney LeRoy George Siddell in People v. Alvarez for filing an opposition containing a fabricated quotation, a citation to a nonexistent case, and misrepresentations of actual case holdings.[1] The court emphasized the heightened concerns when AI hallucinations affect criminal defendants’ rights, noting that such conduct “undermines the integrity of the judicial system” and “damages [attorney] credibility.” [2]
The Alvarez court noted that attorney Siddell had “taken courses regarding artificial intelligence (AI) and was aware that AI could hallucinate cases, but he did not verify the accuracy of any citations.” [3] Despite this knowledge, Siddell admitted he felt rushed and failed to check the case law before submitting documents to the court.[4] The court published its decision to emphasize that “attorneys must make every effort to confirm that the legal citations they supply exist and accurately reflect the law for which they are cited.” [5] The court also highlighted that attorneys cannot delegate verification responsibility to any form of technology, stating unequivocally that this remains “the responsibility of a competent attorney.” [6]
The Alvarez decision reflects a nationwide trend that has accelerated dramatically in recent months. According to tracking efforts by legal researchers, more than 600 cases across the United States have involved attorneys submitting briefs with hallucinated cases.[7] A May 2024 Stanford University RegLab analysis found that although three out of four lawyers plan to use generative AI in their practice, some forms of AI generate hallucinations in one out of three queries.[8] Researchers tracking these incidents report that what began as a few cases per month has escalated to several cases per day, suggesting the problem is intensifying rather than resolving as AI technology advances.[9]
The phenomenon extends beyond isolated mistakes by inexperienced attorneys. Courts have documented instances involving attorneys with decades of practice experience who failed to verify AI-generated content. The recurring nature of these violations across different jurisdictions, practice areas, and attorney experience levels demonstrates a systemic challenge requiring legislative intervention rather than relying solely on case-by-case judicial sanctions.
Core Attorney Requirements
SB 574, introduced by Senator Tom Umberg, chair of the California Senate Judiciary Committee, would add Section 6068.1 to the Business and Professions Code, establishing a few fundamental obligations for attorneys who use generative AI to practice law.[10]
The bill mandates strict confidentiality protections. Attorneys must ensure that confidential, personal identifying, or other nonpublic information is not entered into any public generative AI system.[11] This requirement addresses the documented practice of AI platforms using input data for training purposes or sharing queries with third parties. The practical implication is significant: attorneys cannot simply copy-paste client documents or case-specific information into freely available chatbots like ChatGPT without potentially violating client confidentiality. For attorneys accustomed to the convenience of such tools, this requirement necessitates either investing in enterprise-grade AI platforms with robust data protection agreements or implementing rigorous data sanitization protocols before any AI interaction.
SB 574 requires attorneys to take reasonable steps to verify the accuracy of AI-generated material and to correct any erroneous or hallucinated output.[12] This verification duty extends beyond mere spot-checking. The legislative language imposes an affirmative responsibility to ensure accuracy before relying on AI-generated content in any professional capacity. In practice, this transforms AI from a production tool into a research starting point that requires the same level of scrutiny as work product from an untested summer associate.
Perhaps most significantly, the bill would amend Section 128.7 of the Code of Civil Procedure to prohibit any brief, pleading, motion, or paper filed in court from containing citations that the responsible attorney has not personally read and verified, explicitly including citations provided by generative AI.[13] This provision directly responds to cases like Alvarez, in which the attorney admitted he did not adequately verify AI-generated citations before filing, despite being aware of the risks of hallucination.[14] The requirement eliminates any delegation defense; attorneys cannot claim they relied on technology, staff, or other intermediaries to verify legal authority.
Broader Regulatory Context
SB 574 does not operate in isolation. California has positioned itself at the forefront of AI regulation through multiple legislative initiatives. In 2025, the legislature passed Senate Bill 53, the Transparency in Frontier Artificial Intelligence Act, requiring large AI developers to publish safety frameworks and report critical incidents.[15] Senate Bill 243 regulates AI companion chatbots with safeguards for minors.[16] Assembly Bill 853 addresses disclosure obligations for AI content platforms.[17] This multi-pronged approach reflects California’s strategy of sector-specific regulation rather than comprehensive omnibus legislation.
SB 574 also builds directly on guidance issued by the State Bar of California in November 2023. The Committee on Professional Responsibility and Conduct published “Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law,” which outlined ethical obligations under existing Rules of Professional Conduct.[18] Senator Umberg’s bill essentially elevates that professional guidance into statutory law, providing clearer enforcement mechanisms and eliminating any ambiguity about whether the guidance is merely advisory.
Enforcement and Looking Ahead
Unlike some regulatory frameworks that create new administrative penalties, SB 574 relies on existing enforcement infrastructure. Violations would be addressed through court sanctions under existing procedural rules and through State Bar disciplinary proceedings.[19] This approach avoids creating additional bureaucratic machinery while ensuring that violations carry meaningful consequences, as the Alvarez case demonstrates.
For attorneys, the prudent course is to begin implementing SB 574’s requirements immediately, regardless of the bill’s fate. The principles it codifies reflect fundamental professional obligations that exist independent of statutory mandate. Developing robust internal policies for AI use, establishing verification protocols, and training staff on confidentiality requirements will position firms to adapt quickly to whatever regulatory framework ultimately emerges.
You might also like:
GenAI Prompting for Legal Professionals
The Modern Attachment Revolution: How Cloud Links Are Reshaping Electronic Discovery
The Evolution of Cloud Forensics: Challenges and Solutions in Cloud-Based Investigations
Footnotes
[1] People v. Alvarez, 114 Cal. App. 5th 1115 (2025).
[2] Id.
[3] Id.
[4] Id.
[5] Id.
[6] Id.
[7] Damien Charlotin, AI Hallucination Cases Database (database tracking legal cases where generative AI produced hallucinated citations submitted in court filings), https://www.damiencharlotin.com/hallucinations/ (last visited Feb. 3, 2026).
[8] Id.
[9] “California Issues Historic Fine Over Lawyer’s ChatGPT Fabrications,” CalMatters (Sept. 22, 2025), https://calmatters.org/economy/technology/2025/09/chatgpt-lawyer-fine-ai-regulation/.
[10] S.B. 574, 2025-2026 Reg. Sess. (Cal. 2025).
[11] Id.
[12] Id.
[13] Id.
[14] People v. Alvarez, 114 Cal. App. 5th 1115 (2025).
[15] “California State Senate Bill Aims to Limit Lawyers’ Use of AI,” Governance Intelligence, https://www.governance-intelligence.com/regulatory-compliance/california-state-senate-bill-aims-limit-lawyers-use-ai-0.
[16] Id.
[17] Id.
[18] “Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law,” State Bar of California (Nov. 16, 2023), https://www.calbar.ca.gov/Portals/0/documents/ethics/Generative-AI-Practical-Guidance.pdf.
[19] “California Senate Bill 574 Proposes AI Regulations for Attorneys, Arbitrators,” Reed Smith LLP, https://www.reedsmith.com/articles/california-senate-bill-574-proposes-ai-regulations-for-attorneys-arbitrators/.
About ILS
ILS is the nation’s preeminent Plaintiff-only litigation support provider with expertise in leveraging AI for eDiscovery.We specialize in leveling the playing field for the Plaintiffs’ bar by providing high-quality discovery services to help clients win their cases. Our clients know they are sharing their vital case strategies with like-minded professionals committed and passionate about getting justice for Plaintiffs.
Over the past decade, we have worked on many of the country’s largest and most noteworthy litigations, including mass torts and class actions involving Uber Sexual Assault, Depo Provera, GLP1, Opioids, Social Media Victims, Roundup, JUUL, 3M Combat Earplugs, Google Privacy, Insulin Pricing, Philips CPAP, VW Diesel Emissions, TRT, Takata Airbags, Alex Jones – Sandy Hook, and Actos Bladder Cancer, among many others.
ILS services include consulting, hosting, technical services, forensics & collections, and support.
ILS supports leading platforms, including Everlaw, Relativity, Reveal, Merlin, iConect, and Nebula.
Learn more at www.ilsteam.com.