Answers to Questions from ABA: AI in Action Webinar

Jennifer Riley and I got more questions during the AI in Action webinar than we could answer live. Below are written answers to all of them, including the ones we touched on briefly.

Please verify anything that will drive a real decision and understand that pricing and terms of service change frequently.

If you would like my AI use policy template, you may find it here.

The Big Picture

When thinking about AI use, should the focus for lawyers be both generative AI and agentic AI? So many terms.

Yes. Both matter, for different reasons.

Generative AI is what you get when you type a prompt into ChatGPT, Claude, or Copilot and a response comes back. It produces text, summaries, drafts, and analysis based on your input. You drive each step.

Agentic AI takes a goal and works toward it across multiple steps, often using other tools along the way. Instead of asking it to draft a memo, you use a trigger which begins a multistep process. For example, you have the agent research the issue, pull the relevant authorities, draft the memo, and check the citations. The agent decides what to do next at each step.

For most lawyers right now, generative AI is the daily workhorse. Agentic tools are growing fast and will reshape document review, intake, and routine drafting. In both cases technology is rapidly evolving. The risks with agentic AI are substantially greater than the risks with regular GAI due to the potential for cascading failures.

How does the latest development in synthetic data affect the quality of AI output for legal tasks?

Synthetic data is data generated by another AI model rather than collected from the real world. It has become important because the supply of high-quality human-written legal text on the public internet is finite, and developers need ways to keep training models.

For legal tasks, the effect cuts two ways. Synthetic data can be tailored to fill gaps in training material, like underrepresented areas of law or specialized contract language. That can make models better at narrow tasks. The risk is feedback loops. If a model trains on output from earlier models that were already wrong about a case or doctrine, the errors compound. The polish of the writing tends to mask the substantive problem.

The takeaway is the one I keep returning to. Verify everything that matters. Synthetic data does not change the duty to check the work.

When you say “open” or “public” tools like OpenAI, Claude, and Perplexity, are you referring to the free version or a pro/enterprise version?

When we say “open” or “public,” we mean any tier where you are accessing the tool through a consumer-facing chat interface, paid or free. Also, when the AI is trained on or pulls data directly from the web instead of its own database. The contrast is with enterprise deployments that have signed terms protecting your data. Some enterprise tools are trained with broader data others are trained on closed databases. Lexis, for example, has only been trained with its own database. This minimizes the risk of the tool bringing in unreliable or incorrect information from outside. However, it does not mean the tool will not hallucinate. Lexis does not tend to make up cases like other tools. It does, however, bring up cases that do not stand for the proposition for which Lexis claims, or make inferences that are not correct sometimes.

In Claude, for example, there is a meaningful difference between free, Pro, and Enterprise tiers within a single product. Claude’s consumer plans (Free, Pro, Max) operate under consumer terms. The Team and Enterprise plans operate under commercial terms with stronger default protections and, for Enterprise, Zero Data Retention options through the API.

If you are putting client information into a tool, the question is not free versus paid. The question is which contract governs the data and what that contract says about training, retention, and access. A firm Enterprise account is more protective than a personal subscription, even an expensive one.

Was this the paid ($20) version of Claude or a more robust one?

For the demos in the webinar, I was using Claude Max, which is the $100 a month consumer plan. When I am working with anything client-related, I move to a higher-tier plan or use the API through a tool that has appropriate terms in place. The Max account is fine for showing how the tool behaves but is not the right choice for substantive client work, unless you anonymize the data.

Claude Specifically

Is there a difference between the Pro version and the Enterprise version of Claude?

Yes, several differences that matter for lawyers.

Pro is a consumer plan at $20 a month. It runs under Anthropic’s consumer terms. By default, conversations are retained on the company’s servers for a defined period, and the privacy and training settings are controlled per user. Pro does not include audit logs, role-based access control, single sign-on, or a Business Associate Agreement.

Enterprise is a commercial plan with seats sold per user. Enterprise customers get audit logs, SSO and SCIM, custom data retention controls, role-based access, managed policies, and a Compliance API. HIPAA-readiness with a Business Associate Agreement is available through the Enterprise plan via the sales-assisted process. Anthropic’s published position is that paid plans, including both Team and Enterprise, do not train on customer inputs or outputs.

For solo and small firms, Team is often the right step up from Pro because it adds shared projects and basic admin without the full Enterprise contract. Larger firms or anyone with regulated data should be on Enterprise. Verify current terms at claude.com/pricing and https://trust.claude.com before committing. Also keep in mind that the terms of service and privacy policies can change, so you need to stay on top of any changes.

Do the terms in Claude Pro or business versions give data privacy assurances comparable to Microsoft Copilot Enterprise?

This question was withdrawn during the webinar but it is worth answering because it comes up constantly.

The short answer is that Claude Enterprise and Microsoft 365 Copilot Enterprise are in the same general ballpark on the privacy basics. Both contractually agree not to train on your data, both offer audit logging, both support SSO, and both can sign a Business Associate Agreement. The differences are in the details. Microsoft has a longer track record of enterprise compliance and tighter integration with the Microsoft 365 stack you probably already license. Anthropic’s Enterprise offering provides Zero Data Retention through the API and Compliance API access.

Claude Pro is not in that ballpark. It is a consumer plan and should be treated as such.

I have been impressed with Claude. What can be done to use Claude on more case-specific issues without confidentiality concerns?

A few practical moves.

Move off the consumer Pro plan if you are doing case-specific work. A Team or Enterprise plan brings stronger contractual protections and the data is not used to train the models. Enterprise adds Zero Data Retention options if you go through the API.

Use Projects rather than open chats. Projects let you set instructions and load reference materials in a controlled space, which is easier to audit later than scattered conversations.

Sanitize where you can. Replace party names with placeholders, redact financial figures that do not change the analysis, and strip identifiers when you are doing brainstorming or first drafts. You can paste back specifics at the verification stage.

Document your usage. A short log of which tool was used for which task, what was input, and what was output makes ethics review easier later and is increasingly an expectation under state guidance.

If your use case is regulated, like medical record summaries or anything HIPAA-touching, get the Business Associate Agreement in place before any patient data goes near the tool.

How good is Claude at reviewing PDF documents? ChatGPT has given me poor results, especially with multiple PDFs at a time.

Claude is currently strong at PDF review compared to most chat-based competitors. It handles longer documents well and tends to keep its bearings across multiple files. The 200,000-token context window on the consumer Pro plan, and the larger windows available on higher tiers, mean you can load substantial documents at once.

Three caveats. First, scanned PDFs that are really images of text need OCR before the model can read them well. Run them through Adobe Acrobat or a dedicated OCR tool first. Second, complex tables and forms still produce errors, especially when columns are misaligned. Always spot-check numerical extraction. Third, with multiple PDFs the model can lose track of which document a fact came from. Ask it to cite the source document for each claim and verify.

What was the website again for the Cowork plug-in?

Cowork is at https://claude.com/product/cowork. It is Anthropic’s desktop agent that runs alongside the Claude desktop app and can act on local files and folders. You point it at a folder, give it instructions, and it works through the task. It is now generally available on all paid plans for macOS and Windows. The download page for the desktop app is at https://claude.com/download.

Microsoft Copilot Specifically

Have you tried using a Copilot Agent to sort your emails?

Yes, with mixed results. Copilot can sort, summarize, and triage email reasonably well for high-volume inboxes, especially when you give it a clear set of categories. It is good at flagging messages from specific senders or matching specific topics, drafting replies in a defined tone, and surfacing items that look time-sensitive.

Copilot struggles with judgment calls that depend on context outside the email, like firm priorities, current matter status, or the personal politics of a client relationship. Treat it like a brand-new associate or law clerk. Useful for sorting and summary, supervised on anything substantive.

Can Copilot be used with Gmail?

Yes, with a connector. Microsoft has been rolling out connectors that let Copilot pull from Gmail, Google Calendar, Google Drive, and Google Contacts alongside Microsoft services. Setup happens in the Copilot app’s connector settings, where you authorize Copilot to access the Google account.

The functional issue for legal users is that connecting a Gmail account brings whatever data is in that mailbox into Copilot’s reachable scope. If the Gmail account holds client communications, the same confidentiality questions you would ask about Outlook apply. Get clear on which Microsoft contract governs your Copilot deployment before you connect a Gmail mailbox that contains client data. I do not recommend using free Gmail for any confidential client work. Also, when you use free Gmail, you lose the opportunity to market your own domain name. But Google’s business services have robust privacy protection.

How does Copilot determine when to suggest a rewrite, and what criteria does it use to evaluate clarity, tone, or legal-writing quality?

First, you need to tell Copilot that you want it to review your document. Then, it will suggest changes. Copilot’s rewrite suggestions are generated by the underlying language model based on patterns it learned during training. There is no published rubric and no transparent set of criteria. The model assesses things like sentence length, passive voice, redundant phrasing, and consistency of tone, but the weights it applies are not exposed to users.

The practical implication is that Copilot’s suggestions reflect general writing conventions, not legal writing conventions. It will sometimes shorten sentences in a way that loses a qualifier you carefully added. It will sometimes flatten the formality you want in a brief. Treat the suggestions as a draft editor checking your spelling and rhythm, not as a legal writing reviewer.

You can create a standing order that tells Copilot what sort of conventions you want it to use. You will need to store that standing order somewhere and paste it into Copilot with your prompt each time you begin a new chat.

What can Copilot or Claude help with in a SharePoint data repository?

Both tools can help if you connect them properly.

Copilot has the home-field advantage here because SharePoint is a Microsoft product. With the right licensing, Copilot can search across your SharePoint sites, summarize documents, pull together information from multiple files, and generate drafts that reference internal materials. It respects existing SharePoint permissions, which is a meaningful security feature.

Claude can also work with SharePoint through the Microsoft 365 connector available on Enterprise plans and through Cowork on the desktop. The integration is less mature than Copilot’s native SharePoint access but adequate for many tasks.

If your firm runs on SharePoint and you want AI on top of it, Copilot is the natural starting point. If you want both, you will want to look at your license and have IT look at your security. You also need to think about which tool gets which content.

How do you safeguard against Copilot accessing or deleting records, files, or data beyond the instructions you provided?

There are a few layers of defense within Copilot.

1. Begin with permissions. Copilot operates within the user’s existing access permissions. If a user does not have access to a folder, neither does Copilot acting as that user. Tighten file-level and SharePoint-level permissions before deploying Copilot, not after. This is something to discuss with your IT folks. If you do not have IT folks because you are a solo or small, I suggest finding a good consult who understands Microsoft 365.
2. Sensitivity labels and Data Loss Prevention policies also provide protection. Microsoft Purview lets you label content and apply policies that restrict what Copilot can do with it. This is the most direct technical control.
3. Audit logging. Enterprise Copilot deployments log Copilot actions. Review them periodically.
4. Agent and connector controls. Disable connectors you do not need. Limit which agents can be deployed and by whom.
5. Training. The most common deletion or exposure incident comes from a user telling Copilot to do something they did not understand the implications of. Make sure to train the staff on both Copilot itself and your internal use policy.

Other Tools We Discussed or Were Asked About

Do you know if Clio’s vLex is trustable? It is marketed as on par with Lexis and Westlaw.

vLex is a legitimate legal research platform with a long track record outside the United States, particularly in Spanish-speaking jurisdictions and the United Kingdom. Its Vincent AI tool is widely used and has received favorable evaluations in independent legal tech reviews. Clio’s integration brings vLex’s research capabilities into the Clio practice management workflow.

Whether it is on par with Lexis and Westlaw depends on your jurisdiction and your practice area. For US case law and statutes, Lexis and Westlaw still have deeper headnote treatment, longer-running editorial enhancements, and more comprehensive secondary materials. For straight access to primary law, vLex is competitive. For international research, vLex is often stronger.

I would not rely on any single research platform exclusively. Use vLex through Clio for the speed and integration. Verify anything important on non-AI Lexis or Westlaw or another non-AI research tool that you trust before it leaves your office.

As with any AI-driven research tool, check the cited authorities yourself before relying on them. I have not personally vetted every claim vLex makes about its AI accuracy.

Do you have any experience with Harvey?

I personally do not. What I can tell you is that Harvey is the most well-known dedicated legal AI platform aimed at large law firms and legal departments. It is built on top of foundation models and tuned for legal workflows like document review, contract analysis, and drafting. It has significant adoption among AmLaw 100 firms.

Direct feedback is limited because Harvey is not generally available to small and solo firms. I reached out to Harvey for access, and I was told they do not have time to provide access for testing right now.

Pricing reflects its target market. For lawyers in that target market, the value proposition is integration, custom training on firm materials, and a workflow built around how lawyers work rather than a general-purpose chatbot.

For smaller firms, the practical alternative is to build similar workflows on top of general-purpose tools like Claude or Copilot using Projects, custom instructions, and document libraries. The output is not as polished, but the cost is a fraction of Harvey’s.

Are you familiar with the Atticus Project?

Yes. The Atticus Project is a non-profit effort that builds open-source datasets and benchmarks for legal AI. Their CUAD dataset, which is a labeled set of contracts used for training and evaluating contract review models, is widely cited.

The work matters because legal AI evaluation has been inconsistent and questionable. Vendors publish benchmarks they designed themselves, and the results are not comparable. Open-source benchmarks let practitioners and researchers test claims independently. Atticus’s datasets also help smaller AI projects access training material that would otherwise be locked behind expensive licenses.

I think well of what they are doing. Anyone evaluating a legal AI vendor should ask what benchmarks the model was tested against and whether independent benchmarks like CUAD are in the mix.

What subtitle program are you using? It is fast and accurate.

I checked with the ABA, and they said to reach out to them directly. Click here to email Zambia Brannon Carter.

Off topic: are cases published on Google Scholar? Would AI search them?

Google Scholar publishes US federal and state case law back through the late 1800s for many jurisdictions. Coverage varies. State trial court orders are largely missing. Federal district court opinions are present but not exhaustive. It is a useful free resource but not a substitute for a paid research database for thorough work.

Whether AI can search Google Scholar depends on the tool. Tools with general web access, like Perplexity, ChatGPT with browsing enabled, and Claude with web search, can pull from Google Scholar. The risk is the same risk that produced the Mata v. Avianca embarrassment. Models can fabricate citations or misread cases. If you use AI to search Google Scholar or any open source, verify every citation by opening the case yourself and reviewing it to make sure it stands for the proposition for which it is cited. Also check to see if any quotes the AI cites are actually in the opinion.

Confidentiality and Ethics

You mentioned removing PII, but you are still sharing confidential client information. What terms do you look for in license T&Cs to get comfortable?

This is an important question. The terms I look for, in plain language:

1. A clear statement that the provider does not use customer inputs or outputs to train models. It is preferable that this is the default, but regardless, make sure you check and turn off training.
2. A defined retention period with the ability to set it shorter or to zero. Enterprise tools should support custom retention.
3. Access controls and audit logging. If a privileged user inside the vendor can read your conversations, that is a problem.
4. A Business Associate Agreement available if you handle PHI.
5. Subprocessor disclosure. Most AI vendors use cloud infrastructure under the hood. You want to know who that is and whether the contractual chain holds up.
6. Data residency commitments if you have clients with cross-border concerns.
7. A clear breach notification commitment.

The PII sanitization step is not a substitute for any of the above. It is an extra layer that reduces blast radius if something goes wrong. The contract is the primary protection.

Are there any AI programs you suggest that are secure enough for medical record summaries?

For clinical use cases that touch protected health information, the bar is HIPAA compliance with a signed Business Associate Agreement. Several options exist.

1. Microsoft 365 Copilot under the right enterprise licensing, with a BAA in place. Microsoft has long-standing HIPAA compliance experience.
2. Claude through the Enterprise plan with a sales-assisted BAA arrangement. This is available but requires going through Anthropic’s sales process.
3. OpenAI’s enterprise offerings also support BAAs through dedicated arrangements.
4. Several legal-specific platforms aimed at personal injury and medical malpractice work, including offerings from CaseText (now part of Thomson Reuters) and others, have specific medical record summary tools with HIPAA-compliant infrastructure.

The pattern is the same across the board. The consumer tier of any tool is not appropriate for medical records. The enterprise tier with a BAA is the floor. Verify the current state of any vendor’s HIPAA posture directly with the vendor. This area moves quickly and policies change.

Do you recommend including an AI use clause in your fee or retainer agreements? If so, where can we find a good exemplar?

I recommend addressing AI use somewhere in the engagement, whether in the retainer itself or in a separate disclosure. The disclosure approach is increasingly the norm because it keeps the retainer clean and gives you flexibility to update the AI disclosure as your practice changes.

What to include, at a high level: a brief statement that the firm uses AI tools, the categories of tasks AI may assist with, the firm’s commitment to attorney review of all AI-assisted work product, the firm’s data protection measures, and the client’s option to opt out of AI use on their matter (and the implications, if any, for fees).

For exemplars, the best starting points are the resources published by your state bar’s ethics committee, the ABA’s Standing Committee on Ethics and Professional Responsibility, and the ABA Center for Innovation. Several state bars have issued model language. I would not pull a clause from a random blog post and paste it into your engagement letter without checking it against your state’s rules.

This is a topic where the specific clause should be drafted in coordination with your malpractice carrier as well. They have a view on what they will and will not cover and they are increasingly publishing guidance.

How do you make sure your employees are using AI correctly?

Three things, in this order.

1. A written policy that says what tools are approved, what data is allowed in them, what data is not, and what review is required before AI output goes out the door. Without the policy, training does not stick. Train everyone on the policy itself.
2. Training on the AI tools your firm uses should be practical and recurring. One-time AI training is forgotten by next quarter. A short monthly or quarterly touchpoint, focused on real examples from your firm’s work, keeps the muscle memory fresh.
3. Spot checks. Pick a few matters at random each quarter and review the AI usage on them. Compare what was input, what came back, and what went into the file. This is easier with enterprise tools that log activity.

The biggest mistake I see is treating AI policy as a check-the-box exercise. A signed acknowledgment without follow-up training and review is not protective.

Workflows

Do you use a bank to find language for standing orders? Any sources for language that work well?

By standing orders I am taking this to mean reusable instructions you give the AI. In Claude this is called custom instructions or project instructions. In Copilot it is a similar concept under a different name.

I keep a personal library of prompts and instructions in a plain text or Word file. Each entry has a one-line description, the prompt itself, and a note about when it works and when it does not. It is unglamorous and it is the most useful thing I have.

For published sources, the prompt libraries I find genuinely useful are the Anthropic prompt library at https://docs.claude.com, the OpenAI cookbook, and a handful of legal-focused prompt collections from law school technology programs. The “best prompts” lists that get shared on LinkedIn are usually marketing copy.

The most effective standing order language I have seen is specific. Tell the model what role it is playing, what audience it is writing for, what format the output should take, what to do if information is missing, and what is out of bounds. Vague instructions produce vague output.

The AI itself can help you build your standing orders through a conversation in which you explain what you want the orders to accomplish.

Is it possible to set standing orders for prompts done by an entire organization or firm?

Yes, if you are on the right plan.

Microsoft Copilot Enterprise and Claude Team and Enterprise both let admins configure organization-level instructions and policies. In Claude, you can build shared Projects with instructions and reference materials that the whole team uses. In Copilot, similar functionality exists through Copilot Studio and through tenant-level configuration.

Mechanics matter less than the discipline. Decide who owns the standing instructions, how changes are reviewed, and how often they are updated. Without ownership the standing orders go stale.

Do you have any experience using AI tools to analyze a large body of documents produced in discovery?

Yes. I taught a college level course on e-discovery in which I trained the students on Relativity. The tools that work well for large discovery productions are the platforms purpose-built for ediscovery, like Relativity with its aiR product, Everlaw with its AI Assistant, Reveal, and DISCO. These are designed for the volumes, the chain of custody, the privilege review workflows, and the production formats that real litigation requires.

General-purpose chat tools like Claude or ChatGPT can handle smaller document sets, like a fact pattern review or a focused subset of key documents, but they are not designed for hundreds of thousands of documents. They do not produce the privilege logs, redaction tools, and production tracking you need.

The hybrid pattern I see working: an e-discovery platform for the bulk review and a general-purpose AI tool for deeper analysis of the documents that survive first-pass review.

A strong note of caution. Whatever tool you use, document the methodology. Courts are increasingly asking how AI was used in review, and FRCP 26(g) and the Sedona Conference principles still apply.

Closing

This was outstanding. Please do a part 2.

Thank you. Jennifer and I had a great time presenting and the engagement from the audience was the best we have seen on this topic. We are talking about a follow-up. Watch the ABA CLE catalog. Jennifer and I have been teaching different versions of this program for several years now.

Is there a way to network with the presenters after the seminar?

Yes. You can find me on LinkedIn and reach out directly. Jennifer Riley is also active on LinkedIn. We both welcome questions about AI in legal practice. Click here to email me. You may sign up to receive my blog posts weekly. I am also on Facebook at https://www.facebook.com/jle.jd.

Click here to email Jennifer Riley Jennifer’s website is https://jjrlawfirm.com. 

A Note on Verification

The product details in this post (plan tiers, features, pricing, BAA availability) reflect what was published as of the date of the webinar. Vendor terms change. Before signing anything, verify with the vendor. Specifically:

1. For Anthropic Claude plans, see https://claude.com/pricing and https://trust.claude.com.
2. For Microsoft Copilot, see https://microsoft.com/copilot and the Microsoft 365 service description.
3. For any tool you plan to use with regulated data, get the BAA or DPA in writing before you load the data.