California has become the first state to regulate lawyers’ use of generative AI by statute. Governor Newsom signed Senate Bill 574 on September 30, 2026, and it takes effect January 1, 2027. A California lawyer who mishandles AI will now be violating the Business and Professions Code, not just the Rules of Professional Conduct.
I think other states should be paying attention. If the bar and the courts do not make sure lawyers know how to use generative AI properly, legislatures will decide they have to step in. Traditionally, the legislature does not get involved in regulating attorneys; that is normally left to the courts and the disciplinary system of each state. But if lawyers keep violating the ethical rules and the penalties put forth by the courts and disciplinary bodies do not stop lawyers from violating those rules, it is understandable that legislative bodies might feel the need to step in. Given this, I believe other states will act as well, if we do not see a decrease in the misuse of generative AI.
What Does the New Law Require?
The core of the law is a new Section 6068.1 of the Business and Professions Code. It opens by stating that nothing in the section “shall be construed to abrogate an attorney’s duty to exercise reasonable competence and diligence in the practice of law.” It then provides that “[a]n attorney shall not delegate the practice of law to generative artificial intelligence.”
A lawyer who uses generative AI “to assist in the practice of law” must then do three things.
Use a Properly Confidential Generative AI Tool
First, the lawyer may not enter “confidential, personal identifying, and other nonpublic information” into an AI system unless access to that information is limited, as the statute puts it, “to the attorney and persons authorized by the attorney under obligations to protect the confidentiality of the information.” The definition of personal identifying information is broad. Along with the obvious items, like Social Security and driver’s license numbers, it includes dates of birth, “[m]edical or psychiatric information,” “[f]inancial information,” and the “[a]ddresses and phone numbers of parties, victims, witnesses, and court personnel.”
The reality is that many attorneys, especially those in solo and small firms, find the appropriate level of generative AI tools to be too costly and turn to consumer versions of tools such as ChatGPT or Claude, failing to realize that those account levels are not appropriate for confidential client information. The first level of tool that might be acceptable, which I have analyzed in my posts on the privacy policies of both tools, is the first commercial level of account, called Team or Business. However, even that level of account may not be acceptable for certain types of data. For example, I do not believe that it is acceptable to enter data protected by HIPAA in accounts that do not specifically have Business Associate Agreements. It is critical for attorneys who use generative AI to review the terms of service and privacy policies of any AI tool they use and make certain that there is an appropriate level of privacy protection for the type of data they are entering. If not, they must choose a different tool, a different level of account, or redact the protected data.
Verify All Cases and Factual Information
Second, the lawyer must “[t]ake reasonable steps” to “[v]erify the accuracy of generative artificial intelligence outputs, including, but not limited to, the accuracy of all case and statutory citations” and to “[c]orrect any erroneous or hallucinated output in any material used by the attorney.”
How should a lawyer verify data accuracy? For cases and statutes, the best way to verify the accuracy is to actually read the original document. Never use one generative AI to verify another. While it is certainly acceptable to have one generative AI look at another’s work, to make sure the information is correct requires reading the original factual source. AI is capable of providing fabricated citations, false propositions, or incorrect citations. The only solution for this problem is to use a proper non-generative AI database such as Lexis, Westlaw, or free databases such as CourtListener or Google Scholar, or similar databases. On occasion a case will not be in a database, so it is necessary to go to the correct court’s website or PACER to locate the information. Similarly, summaries of cases, trial transcripts, depositions, and other documents can be incorrect. Sometimes generative AI leaves out something important. More worrisome, generative AI can add witnesses that do not exist or false statements. This happened to a New Mexico lawyer who used ChatGPT to summarize a trial transcript.
Disclosure Requirements
Third, the lawyer must “[d]isclose the use of generative artificial intelligence to the court for all documents submitted to the court” and “consider whether to disclose” AI use when it creates content provided to the public. Does that mean every filing by a lawyer who uses AI anywhere in the practice needs a disclosure, or only filings AI helped produce? The statute does not say. California courts will have to work that out.
Different courts have different requirements. Here in Pennsylvania, some jurisdictions or individual courts have specific requirements while others rely on the Rules of Professional Conduct. It is critical to check every court to make certain you are following any disclosure requirements properly. There is not, as of this writing, a Commonwealth wide disclosure requirement in Pennsylvania.
Personally Verify All Citations
The law also amends California’s sanctions statute, Code of Civil Procedure Section 128.7. A filing “shall not contain any citations that an attorney responsible for submitting the pleading has not personally verified, including any citation provided by generative artificial intelligence.” That requirement applies to every citation, whether or not AI was involved. Arbitrators get their own rule as well: “An arbitrator shall not delegate any part of their decisionmaking process to any generative artificial intelligence tool.” The bill also amends Section 6173 of the Business and Professions Code to change how the State Bar handles complaints against the alternative dispute resolution providers it certifies.
This requirement to personally verify all citations can be challenging. Many lawyers rely on paralegals or associates to write documents that will be filed and to verify citations before they are filed. For many years, some senior associates and partners have relied on the accuracy of citations without checking them. Under both the California statute and the ethical rules, this is no longer possible. On the other hand, those acting as outside counsel to businesses and insurance companies are constantly pressured to reduce bills. How are we to balance these competing requirements?
Under the statute, a filing may not contain any citation that “an attorney responsible for submitting the pleading has not personally verified.” We already had this obligation under our ethical rules, but busy attorneys frequently skip this step, as is proved by the many attorneys who have gotten sanctioned when they signed a brief or supervised lawyers who signed a brief. Verifying all citations and factual information is a good habit regardless of whether those involved are required to follow a statute or just the ethical rules. The best process for meeting this requirement is for those who draft the filing to provide a link or pdf of the specific case or document with details about the information and quotations highlighted. This will allow the other signing or supervising attorneys to quickly check any citation or factual detail that is contained within the document.
Every Firm Needs a Written AI Policy
Someone needs to be responsible for viewing those terms and keeping them up to date. All firms, even solo firms, need a written policy to spell out these requirements and what lawyers are and are not allowed to do with generative AI. These policies help to remind all attorneys and staff of the limitations of generative AI, what can be entered in each tool, and which tools are acceptable, along with the firm’s verification process. While a solo attorney may not think they need a policy, if they have staff, the staff need the reminder, and the policy can be helpful if the attorney gets called out because of a seeming hallucination or problem with a filing. Not all errors are due to AI, and a written policy will show a court and disciplinary authority that the lawyer has thought through the issues in a responsible way. If you do not have a policy and would like a place to start, feel free to use my template.
Did Lawyers Need a Statute?
Most of what SB 574 requires, lawyers were already required to do. The duty of competence did not stop applying when ChatGPT arrived, and neither did the duty of confidentiality. ABA Formal Opinion 512 addressed generative AI in 2024. So did Pennsylvania’s Joint Formal Opinion 2024-200. The courts have been sanctioning lawyers for hallucinated citations for years now, under rules that were already on the books. While hallucinated citations date back to 2022, when ChatGPT became publicly available, errors in citations are hardly new. We have always had this obligation. It is simply that failure to verify citations in the past or factual mistakes were generally treated as errors as opposed to ethical violations.
The problem, despite the substantial education provided by many bar associations, is either that too many lawyers do not understand how these tools work or that they are simply too busy and skip verification steps. A lawyer cannot verify output if they do not understand that a tool can invent a case that looks real. As for being busy, I certainly understand. But if an attorney is busy, the solution has to be to slow down a little bit to double-check, given the potential risks involved. In addition, a lawyer cannot keep client information out of an unrestricted system if they have never read the terms of the product they are using, or do not know which version of a tool the firm is paying for. Again, this is a problem of time, but it is necessary for someone at the firm to review the terms of service and privacy policies of the tools they use. In the case of a solo, that means the only attorney available.
Is the Problem Really That Big?
I will note that in the scheme of things, the number of cases with hallucinated citations is quite small. The majority of cases involving hallucinations tracked online are actually from pro se parties and not attorneys. But there are enough cases involving attorneys that we must take note and do something to minimize them. It is unfair to clients and wasteful to the system to have attorneys filing cases with incorrect information. It is also unfair to clients and opposing parties when generative AI that is not properly confidential is used. Attorneys already have issues with public relations and the problem with hallucinated cases is known to the public. This does nothing to help how the public sees our profession.
Training Is the Better Answer
Given the fact that the rules already address the issues covered in the statute, I continue to believe the answer is training. I also believe that the training has to be both practical and ethical. Lawyers need to learn how the tools work, how to check what comes out of them, and how to read a vendor’s terms, along with the ethics rules that govern all of it. An hour of abstract AI ethics once a year is not enough. The training must be continuous, and the attorneys must seek it out and also make sure their staff are properly trained.
If states provide proper training now, they give lawyers a real chance to meet the obligations they already have. If they do not act to reduce the problems, they may find a legislature has written those obligations for them, in language that courts will spend years interpreting.
Courts and Disciplinary Bodies Have a Role
I also think that both the courts and the disciplinary bodies need to be involved in enforcing the ethical rules. Some courts have handed down substantial fines, others barely a slap on the wrist. Rarely have disciplinary bodies become publicly involved. They need to do so now to make it clear that continuing behavior of this type is not acceptable. While I do not believe lawyers need to be suspended or disbarred due to their first problem with generative AI, I do believe an admonition of some kind, even a private one, might stick better, especially since we have seen some attorneys have more than one filing with hallucinated cases.
Pennsylvania Is Doing the Work
Pennsylvania has been doing this work. The Pennsylvania Bar Association and the Pennsylvania Bar Institute have taken lawyers’ use of AI seriously, and the PBA and Philadelphia Bars’ joint opinion gave Pennsylvania lawyers guidance early. As I wrote in When AI Cites AI: The Misinformation Loop, the claim that Pennsylvania requires AI disclosure in all court submissions came from vendor blog posts, not from any Pennsylvania rule. Pennsylvania got here through guidance and education rather than a statute. But we are not perfect. We have our share of lawyers using hallucinated citations and I have no doubt that some attorneys are using generative AI that is not confidential enough. As a result, if we do not act firmly, we will no doubt see additional state governments feel the need to step in.
The AI Academy
Here in Pennsylvania, the Pennsylvania Bar Association and its continuing legal education arm, the Pennsylvania Bar Institute, are looking to implement an AI Academy. This concept was brought to the PBA and PBI by the PBA’s Technology, Cybersecurity and Data Privacy Committee. The AI Academy will provide a series of pre-recorded modules that educate attorneys on different aspects of AI. Upon completion of each module, the attorneys will be required to pass a test. Once the attorney successfully completes all modules, they will be provided a certificate of completion. Given the rapid changes in AI, that certificate will expire each year, requiring the attorney to update their education for a new certificate. While the Academy will not guarantee any specific ability in AI, it will show that, at the least, the attorney has received proper education in the ethical obligations and practical use of generative AI in their work. I cannot tell you when the Academy will be ready, but likely sometime in 2027. Please understand that this is an unofficial announcement, and likely the first time anyone outside the Committee, PBI or PBA has heard about this concept, and there may be changes in the details before it is implemented.
Conclusion
I would rather see lawyers learn to use these tools well than see fifty legislatures decide how they may use them. But if we do not act quickly to stem the flow of generative AI misuse, that is exactly what will happen. I encourage all bar associations of all types as well as the disciplinary bodies of all jurisdictions to step in and remind attorneys of their ethical obligations. That way we can protect both the public and the legal community from further problems with generative artificial intelligence.
Leave a Reply