Attorneys filed ChatGPT-hallucinated case citations to federal court
Attorneys filed a federal brief with six cases ChatGPT fabricated, reaching the court docket without independent approval.
What happened
In 2023, attorneys in the Southern District of New York filed a brief citing six cases ChatGPT had fabricated. When the citations were questioned, they submitted fake case excerpts. Judge Castel found bad faith and sanctioned the attorneys and their law firm $5,000 under Rule 11. The drafting agent and the filing actors were the same parties, with no separate party accountable for what reached the federal court docket.
What the agent did
ChatGPT generated a legal brief with fabricated case citations; the attorneys controlling the system then filed it to the federal court docket
The irreversible effect
The false brief was submitted to a federal court docket, becoming a binding representation to the court under Rule 11, resulting in judicial sanctions and legal liability that could not be quietly withdrawn or corrected without court action
Root cause
The drafter and the filing actor were identical: no segregation between reversible draft actions and the irreversible filing action. A separate approval gate requiring an independent human reviewer to sign off on the exact filing version before submission would have created accountability and an opportunity to catch the fabrication.
How a maker-checker control would have refused it
Two refusals block the incident: (1) The drafting role holds only draft and lookup grants; filing is not granted, so mata-court-file@1 is refused with skill_not_granted before anything reaches the docket; (2) Even if a role held the filing grant, mata-court-file@1 is marked high-risk, so the proxy refuses it with high_risk_requires_gate and requires it to run through a governed flow with a preceding approval gate where a named supervising attorney must explicitly sign off on the exact brief version before filing.
Runnable reproduction
This incident ships as a runnable scenario in the open-source repository. Point the enforcement engine at the policy and watch the action get refused, with the refusal written to a signed audit record.
examples/mata-v-avianca-fabricated-citations-filed
Accuracy and corrections
This entry describes a publicly reported incident and is compiled from the primary sources listed above. Where an account is a legal allegation rather than an established finding, the entry labels it as such. Summaries can still contain errors. If you can document a correction, email hello@makerchecker.ai and we will review and correct it, with the change noted, within 14 days.
See it for yourself
Reading is one thing. Watch it block an agent.
One command starts the demo: an agent stopped from signing off its own work, and the signed evidence file an inspector can check for themselves.
Designed against the rules your auditors already enforce.