In previous posts, we’ve explored some fundamental problems with artificial intelligence in legal contexts—particularly AI’s troubling tendency to cite cases that simply don’t exist and to mischaracterize the holdings of cases that do.
But in fairness, the problem does not lie solely with Artificial Intelligence. AI is a great tool and it can definitely help you put together legal documents. But.… you have to understand its limitations.
Court rules seem like they should be perfect for AI. They’re written in clear language, organized systematically, and publicly available. What could go wrong? Everything, as it turns out.
The problem is that legal rules exist within layers of interpretation that fundamentally transform their meaning. A rule might state one thing plainly, but decades of case law, Supreme Court decisions, and circuit interpretations create an entirely different operational reality. AI systems, even sophisticated ones, struggle profoundly with this gap between text and application.
FRCP 8: A Case Study in Judicial Reinterpretation
Consider Federal Rule of Civil Procedure 8(a)(2), which governs pleading requirements in federal court. The rule itself seems straightforward:
“A pleading that states a claim for relief must contain… a short and plain statement of the claim showing that the pleader is entitled to relief.”
Read that language. “Short and plain statement.” It sounds simple, almost casual. An AI analyzing this text would likely conclude that minimal detail is required—just enough to put the defendant on notice of the claim.
But that’s not what Rule 8 means at all.
This is a typical example of IKYTYUWISBWISINWIM
“I know you think you understood what I said, but what I said was not what I meant!”
In two landmark decisions—Bell Atlantic Corp. v. Twombly (2007) and Ashcroft v. Iqbal (2009)—the Supreme Court fundamentally rewrote what Rule 8 requires, without changing a single word of the rule itself.
The Court established that complaints must now contain “enough facts to state a claim to relief that is plausible on its face.” This plausibility standard requires plaintiffs to plead factual content that allows the court to draw the reasonable inference that the defendant is liable. Conclusory statements won’t suffice. Mere consistency with liability isn’t enough. The facts must suggest that the claim is plausible, not merely possible.
This is a dramatically higher bar than “short and plain statement” suggests. Lower courts have dismissed countless complaints that would have easily satisfied the plain text of Rule 8 but failed the judicially-created plausibility standard.
Here’s where AI fundamentally breaks down:
Textual analysis fails. An AI reading Rule 8 has no inherent way to know that “short and plain statement” actually means “factually detailed plausibility showing.” The rule’s text hasn’t changed since 1938, but its meaning was revolutionized in 2007-2009.
Context requires judgment. Understanding how Twombly and Iqbal apply requires grasping nuanced distinctions about what makes a claim “plausible” versus merely “possible”—distinctions that experienced attorneys debate and that vary by circuit, claim type, and factual context.
The law is living. These interpretations continue to evolve. District courts apply Twombly-Iqbal differently. Circuit courts emphasize different factors. An AI trained on cases from 2010 would miss crucial developments from 2024, and even a current AI can’t predict how a particular judge will apply the standard to novel facts.
Rule 8 isn’t an outlier—it’s the norm. Throughout federal and state procedure, substantive law, evidence rules, and statutory interpretation, what rules say and what rules mean often diverge significantly. These gaps emerge from:
– Supreme Court interpretations that add requirements not in the text
– Circuit splits that create geographical variations in meaning
– Evolving standards that shift over time
– Context-specific applications that depend on claim type, procedural posture, or factual scenarios
An AI might tell you what Rule 8 says. It might even cite Twombly and Iqbal. But can it reliably advise whether your specific complaint will survive a motion to dismiss before a particular judge in a particular circuit? The track record suggests not.
Legal practice isn’t an arena where “usually right” is good enough. A missed nuance in pleading standards means dismissal. A misunderstood procedural rule means waiver. A failure to grasp how courts actually apply seemingly clear text means malpractice.
When we’ve already established that AI hallucinates non-existent cases and mischaracterizes real ones, adding the complexity of judicially-transformed rule meanings creates a perfect storm of unreliability. The technology simply isn’t ready for the interpretive sophistication that legal practice demands.
AI tools may eventually have a supportive role in legal research and practice, but that role must be carefully circumscribed and heavily supervised. The gap between what court rules say and what they mean—exemplified perfectly by Rule 8’s transformation from “short and plain” to “plausible and detailed”—reveals a fundamental limitation in current AI systems.
Until AI can reliably navigate the layers of interpretation, evolution, and judicial gloss that define what legal rules actually require, attorneys and litigants who rely on it do so at their peril. The law is not just text—it’s centuries of interpretation, application, and refinement. That’s not a dataset problem. It’s a fundamental challenge to whether AI can truly “understand” law at all.