Blog

In AI-Powered Lawyering Study, Vincent AI creates far fewer hallucinations than OpenAI’s o1-preview – and even fewer than humans.

Miami, FL (March 5, 2025) — A new benchmark study from the law schools at the University of Minnesota and the University of Michigan found that Vincent AI from vLex significantly enhances the quality of legal work and significantly boosts productivity. In a first, the study also found that Vincent AI hallucinated significantly less than OpenAI’s o1 reasoning model, but also even less than upper-level law students in the study.

The study, “AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice,” compared legal tasks performed by upper-level law students using Vincent AI and OpenAI’s o1 reasoning model. The results provide what authors call “the first empirical evidence, to our knowledge, that AI tools can consistently and significantly enhance the quality of human lawyers’ work across various realistic legal assignments.”

Results from the report found that “AI assistance significantly boosts productivity in five out of six tested legal tasks, with Vincent yielding statistically significant gains of approximately 38% to 115%.”

The research team identified substantially fewer hallucinations in assignments completed with Vincent AI than in assignments completed with o1-preview and even fewer hallucinations than in those completed without any AI assistance at all, concluding: “the data suggest that RAG technology, such as that used by Vincent AI, does indeed reduce hallucinations. In fact, we identified fewer hallucinations in assignments completed with Vincent AI (3 total) than in those completed without any AI assistance at all (4). By contrast, assignments completed with o1-preview exhibited a substantially higher number of hallucinations (11).”

“For any doubters, this is the third study in a row that illustrates that Vincent AI is ready for primetime,” says Ed Walters, Chief Strategy Officer of vLex. At its AI Smackdown on Feb. 8, the Southern California Association of Law Libraries compared three leading AI research platforms and marked Vincent as having the most depth. The Vals Legal AI Report, released last week, found that Vincent performed with equivalent or higher accuracy than human lawyer benchmarks in four out of the five tasks it tested. And this week’s report on AI-Powered Lawyering finds that Vincent creates measurable improvements on quality and productivity, and hallucinates even less than human researchers.

The study concluded that, “A clear trend in this unblinded qualitative review was that participants who used an AI tool generally produced writing that was easier to read and more polished than those who did not. Their sentences were more concise, their paragraphs flowed more smoothly, and their overall structure presented information in a more coherent and user friendly manner. Additionally, these submissions were largely free of typos, comma splices, and other distracting errors.”

“These results validate not only that Vincent increases the speed and quality of legal work,” Walters added. “But it also adds proof that Vincent’s transparency safeguards protect users from hallucinated sources. The tool is only getting better, and lawyers who use it are already outperforming lawyers who don’t.”

About vLex

vLex is a global legal intelligence and legal AI pioneer empowering legal professionals to work smarter. vLex has united industry leaders Fastcase, Docket Alarm, and NextChapter to transform AI-driven legal services and provide the most extensive collection of legal and regulatory information worldwide, all on one platform. Trusted by more than two million lawyers, law librarians, researchers, government departments, and law schools worldwide, vLex offers comprehensive and intuitive access to the law in more than 200 jurisdictions through an intelligent, AI-powered legal research platform.

With nearly three decades of experience in legal tech and access to over one billion legal documents, vLex developed Vincent AI - a comprehensive AI-driven workflow platform that addresses the challenges faced by modern legal professionals. Unlike general-purpose AI tools prone to hallucination, Vincent AI is powered by vLex's vast library of structured legal data, providing added confidence with citations to verified authoritative sources. Vincent AI’s curated, pre-built workflows have each been carefully crafted to help you accomplish tedious tasks, work smarter, and do more with less. Learn more at vlex.com.