When I teach about AI, I use a ladder of risk. The lowest rungs are the least risky tasks. The highest are the most. The greater the risk, the greater the care required.
Why Low Risk Does Not Mean No Risk
Every day that I use AI, one of the AIs will hallucinate or disobey my instructions. For example, I put in Claude’s standing orders not to use em dashes. In response to this standing order, Claude started using two dashes instead. When I asked it if it was using two dashes because I told it not to use em dashes, Claude responded that yes, it knew it shouldn’t have done it and was sorry. I adjusted my orders to tell Claude not to use two dashes, and not to try to get around my instructions. Yet, from time to time, Claude still uses two dashes or an em dash. If I ask it about it, it always apologizes, but that failure to obey my orders is a serious problem. Especially for those who do not verify AI outputs. Anyone who follows AI use in the practice of law knows that AI has a tendency to hallucinate cases and quotations or misstate the holding. I have also reported on the agentic AI which deleted the databases of a company in 9 seconds. These issues explain why it is necessary to verify all output from AI and why a draft from AI can never be treated as the final result.
The Ladders
Here are two ladders I use in my presentations. The first is for attorneys. The second is for expert witnesses. The links under each image are to a document that contains additional information for attorneys and expert witnesses, respectively.
As noted above, in all cases, it is necessary to include verification in the workflow. The complexity and level of that workflow depend on the level of risk inherent in the task. The greater the risk, the more verification is required. If you used AI for legal research in a brief, verify that the case exists, that the quotes are accurate, and that the case actually stands for the proposition you cite it for.
I hope you find these ladders useful in understanding the levels of risk inherent in using artificial intelligence. If you need assistance with training or integrating in AI into your office, feel free to reach out.