What the Court Record Actually Shows About Legal AI

The risk conversation around legal AI is usually run on anecdote — the viral story of the lawyer who cited cases that didn't exist. It doesn't have to be. There is now peer-reviewed testing of the tools and a systematically maintained database of the incidents. The picture they paint is more precise, and more useful, than any single horror story: the danger isn't "AI in law firms." It's unreviewed AI output in tribunal-facing work.

The tools hallucinate — including the professional ones

Stanford researchers, publishing in the Journal of Empirical Legal Studies (Magesh et al., 2025), tested the legal-research platforms marketed as hallucination-free across 202 preregistered queries. The results: Lexis+ AI hallucinated on 17% of queries, Westlaw's AI-Assisted Research on roughly 33%, and general-purpose GPT-4 on 43%.

Two recency cautions, in both directions. The tests were run in 2024 on the tool versions of that moment, and models improve quickly — so treat the exact percentages as a dated snapshot and re-test the current version of any tool before you rely on it. But what hasn't aged is the failure mode: professional-grade tools produce confident, false legal assertions, and every newer model still does it at some rate. The numbers move; the need for review doesn't.

The incidents concentrate in firms like yours

The Charlotin AI Hallucination Cases database — a running tally of documented incidents — grew from 87 cases in May 2025 to roughly 1,700 by July 2026. Stanford's analysis of the U.S. attorney incidents found that solo practitioners accounted for 50.4% and firms of 2–25 attorneys for another 39.5% — together, about 90%.

That's not because small firms are careless. It's because they're precisely where the two ingredients meet: individual, ungoverned use of consumer tools, and the absence of a second chair to catch the error before it's filed. Big firms have review layers. Solo and small firms often don't — and the record shows it.

One honest caveat: the database counts caught incidents, not rates. There's no denominator, so it can't tell you how often AI errors slip through undetected. What it does establish — concentration and trajectory — both point the same way.

The penalties are real, and they're escalating

For a long time the downside was reputational. Now it's on the docket. The largest verified U.S. AI-hallucination penalty to date is roughly $110,000, imposed on two attorneys in an Oregon federal court in early 2026 — over 15 fabricated cases and 8 fabricated quotations in their filings.

And in July 2026 the stakes crossed from money to licences. Ontario's Law Society Tribunal handed a Toronto lawyer a six-month licence suspension — reported as the first by a Canadian law society for AI misuse in court filings — over a factum built on non-existent, AI-generated case law. The tribunal's sharpest language wasn't aimed at using the tool. It was aimed at what came after: denying the AI use and shifting the blame. The error itself was recoverable at the review stage. The unreviewed filing, and the denial that followed, is what ended a thirty-year clean record.

The distinction that turns risk into a system

Put the evidence together and it doesn't argue for banning AI. It argues for governing it at exactly one point: before anything AI touched goes in front of a court, a human reviews it. That single rule — "a person stands behind every consequential output, under their licence" — is what separates the firms using AI as leverage from the ones that end up in the database.

Getting there isn't a policy memo. It's an approved-tool list, a written classification of where AI may and may not run, and a review habit that holds under deadline pressure. Those are learnable, and they're cheaper than one sanction.

Get the full field guide. The AI Your Firm Never Approved turns this evidence into a governance system — the approved-tool list, the function-by-function map of where AI belongs first, and an eight-step, 90-day plan. Sourced, footnoted, and written for managing partners, not IT. Download the free eBook →