Much has been written about the risks of lawyers relying on generative artificial intelligence (Gen. AI) to draft briefs—especially when unchecked citations turn out to be hallucinated. But lawyer briefs are not the only documents that demand scrutiny. As the cases below show, lawyers must also carefully review their experts’ affidavits, declarations, and reports before submitting them to other parties or the court.
In Kohls v. Ellison, No. 24-CV-3754, 2025 WL 66514 (D. Minn. Jan. 10, 2025), plaintiffs challenged a Minnesota law banning “deepfakes” aimed at harming political candidates or swaying elections. To defend the law from a preliminary injunction, the Minnesota Attorney General Ellison submitted an expert declaration from Stanford Professor Jeff Hancock, explaining the threats posed by AI and deepfakes. The ironic twist: Hancock’s report—drafted with help from a large language model (LLM)—cited two nonexistent academic articles and misattributed a third.
The attorney general’s counsel quickly apologized, claiming they were unaware of the false citations. While the court appreciated the prompt admission, it reminded counsel that Rule 11 imposes a “personal, nondelegable responsibility” to ensure the accuracy of everything filed. The court went further, suggesting that lawyers may need to ask their experts directly if they used AI and how they verified any AI-generated material. Because the errors undermined Professor Hancock’s credibility, the court excluded his testimony when ruling on the injunction.
A similar Gen. AI mishap surfaced in Concord Music Grp., Inc. v. Anthropic PBC, No. 24-CV-03811-EKL, 2025 WL 1482734 (N.D. Cal. May 23, 2025), a copyright dispute in which the parties sparred over the required sample size of LLM “prompts” and “outputs” to be produced. Both sides submitted expert declarations, but during the hearing, plaintiffs moved to strike one of the defense declarations because it cited a nonexistent article with co-authors who had never actually collaborated.
The defendants called it an “honest citation mistake,” admitting they’d used an LLM to format citations, which led to the error. Although the real article was properly linked elsewhere, the court didn’t mince words, calling the mistake “a plain and simple AI hallucination” and faulting the defense for missing it in their “manual citation check.” As a result, the court struck the problematic paragraph and noted the incident undermined the credibility of the expert’s entire declaration—a factor in its final decision.
Even when generative AI does not invent authority, courts may still balk at expert opinions that lean on it. In Matter of Weber as Trustee of Michael S. Weber Trust, 85 Misc. 3d 727 (N.Y. Sur. 2024), the court tossed out an expert’s valuation as unreliable and speculative—and further noted that the report’s reliance on Gen. AI deserved special scrutiny. On the stand, the expert admitted he’d used an LLM chatbot to double-check his calculations, but couldn’t recall his prompts, explain the chatbot’s sources, or clarify its methods. He also did not testify about fund fees or tax implications.
Citing the Frye standard for scientific evidence, the court stressed that AI-generated evidence must be generally accepted in its field. Given Gen. AI’s evolving and uncertain reliability, the court held that attorneys must disclose AI use in expert materials and that such evidence should go through a Frye hearing before being admitted—with the court deciding the hearing’s scope.
This is not to say experts are banned from using Gen. AI. In fact, the Kohls court emphasized it did not fault Professor Hancock for using AI in his research, noting AI’s potential to transform legal practice for the better. The real issue is that attorneys and experts cannot substitute independent judgment with AI-generated answers.
Some courts have allowed expert testimony that involved Gen. AI. In Ferlito v. Harbor Freight Tools USA, Inc., No. CV 20-5615, 2025 WL 1181699 (E.D.N.Y. Apr. 23, 2025), the court refused to exclude an expert who used an LLM to double-check his findings. Crucially, he relied on his decades of experience to write the report first, then used AI only to confirm what he had already concluded.
The rise of generative AI offers new possibilities for expert analysis—but also new pitfalls. As these recent cases show, when experts lean too heavily on AI, credibility may quickly unravel. Courts are scrutinizing not just the substance of expert opinions, but how those opinions are formed. If experts simply echo AI-generated content without exercising independent judgment and careful verification, their testimony risks being tossed—and their reputations damaged. The lesson is clear: AI may assist, but it cannot replace the rigor, experience, and accountability expected from expert witnesses in the courtroom. Attorneys must be vigilant as to whether their experts used Gen. AI and to check the expert’s work-product carefully before producing their reports.1
1 The U.S. Judicial Conference’s Advisory Committee on Evidence has proposed two amendments to the Federal Rules of Evidence to address the authenticity and reliability of digital evidence, including AI-generated evidence. A proposed amendment to FRE 901 (901(c)) would allow a party to challenge – with sufficient support – evidence as fabricated by Gen. AI. If the challenging party meets its burden, the proponent of the evidence must show that the evidence is more likely than not authentic. Proposed new FRE 707 would require the proponent of machine-generated evidence demonstrates to the court that the evidence meets reliability criteria mirroring the standards for expert witnesses. Neither of the proposed amendments have yet been adopted.