Generative AI and Legal Work

Abstract

This report synthesizes controlled experiments, empirical evaluations, professional guidance, and risk-management research concerning generative AI and legal work. It distinguishes evidence about particular tasks and systems from broader claims about the legal profession. It is an evidence review, not an original experimental study.

Executive summary

The empirical record does not support a single, general claim that generative artificial intelligence makes legal work “better.” It supports a narrower and more useful conclusion: AI can materially improve the speed of some legal tasks, and newer reasoning and retrieval-augmented systems can improve some dimensions of work quality, but the result depends on the task, the system, the user, the source environment, and the review process.

The practical implication is that legal organizations should evaluate AI at the level of the workflow, not the product category. The appropriate unit of analysis is a defined legal task performed by defined users with identified inputs, authorities, review requirements, and consequences of error.

1. The evidence is conditional, not categorical

Public discussion of legal AI is often organized around two competing assertions: AI will radically transform legal practice, or AI is too unreliable for serious legal work. Neither assertion is specific enough to guide an implementation decision.

“Legal work” includes factual synthesis, issue spotting, legal research, chronology construction, contract review, drafting, negotiation, counseling, judgment, advocacy, supervision, and administration. These activities differ in their information requirements, verifiability, tolerance for error, and dependence on professional experience.

Likewise, “AI” may refer to an ungrounded public chatbot, a model supplied with a closed set of documents, a retrieval-augmented legal research system, or a reasoning model operating within a structured workflow. The observed effect is consequently a property of the complete system.

This formulation explains why different studies can generate different findings without contradicting one another. A summary can fail to improve performance while document highlighting succeeds. An older general-purpose model can produce speed gains without better legal analysis, while a newer reasoning model improves both. A user with access but no training can behave differently from a user given a clear method and permitted use case.

For legal leaders, this changes the implementation question. Enterprise licensing is not the intervention. The intervention is the redesigned, governed workflow in which a tool is used.

2. Productivity gains are real, but not uniform

Nielsen, Skylaki, Norkute, and Stremitzer conducted an experiment with 206 law students performing a private-law task. Participants received either no machine assistance, AI-generated summaries, AI-generated highlighting, or both. Highlighting reduced task-completion time by approximately 30% without reducing measured quality. Summaries, by contrast, did not improve performance. The combined condition improved efficiency, but less than highlighting alone.

This result rejects the idea that more generated content necessarily creates more value. The most effective intervention did not ask a model to replace legal analysis. It changed the information interface: legally relevant text was easier for the user to locate and process.

Choi, Monahan, and Schwarcz conducted a randomized controlled trial in which law students completed realistic legal tasks with or without GPT-4. AI assistance produced large and consistent speed improvements. Quality gains were smaller, less consistent, and concentrated among lower-performing participants when gains appeared at all. Speed and quality therefore moved independently.

Research outside law reinforces the distinction. Noy and Zhang’s preregistered experiment involving 453 professionals performing writing tasks found that ChatGPT reduced completion time by 40% and increased quality by 18%. The result demonstrates potential in knowledge work, but should not be mechanically transferred to high-stakes legal analysis. Legal work adds authority, confidentiality, candor, supervision, and professional-responsibility requirements.

What the productivity evidence supports

• AI can reduce the time required for defined legal and professional tasks.
• The magnitude of the gain depends on task and interaction design.
• A faster workflow is not necessarily a better workflow.
• Generating prose is not the only, or always the best, form of assistance.
• Productivity should be measured against prompting, correction, verification, escalation, and rework.

3. Quality depends on model capability and source design

Schwarcz and colleagues assigned upper-level law students to realistic lawyering tasks using either a retrieval-augmented legal system, an early reasoning model, or no AI. Both AI conditions significantly increased speed. Reported productivity gains ranged from 50% to 130% in five of six tasks.

Quality improved on at least four tasks. The nature of the improvement differed by system. The retrieval-augmented tool primarily improved clarity, organization, and professionalism. The reasoning model produced broader improvements, including stronger analytical depth on demanding tasks. The retrieval system generated fewer nonexistent citations than the reasoning model, although neither system removed the need for review.

4. Accuracy requires a source and verification architecture

Dahl, Magesh, Suzgun, and Ho systematically tested public-facing language models on specific, verifiable questions about randomly selected federal cases. The models hallucinated between 58% and 88% of the time. They also frequently accepted false premises embedded in questions and were not consistently able to identify their own uncertainty.

The study measured open-domain legal knowledge in 2023-era public models. It does not establish that every contemporary model will hallucinate at the same rate, nor does it measure a well-designed closed-document workflow. Its durable lesson is methodological: fluency, confidence, or general benchmark performance cannot establish the reliability of a legal proposition.

Source-grounded systems can make verification more efficient, but they do not eliminate it. A citation supplied by a retrieval system is an inspectable claim, not proof that the proposition is accurate, complete, or controlling.

5. Training is part of the technology

The productive effect of AI depends partly on whether users adopt it and know how to apply it. Chen and Bao’s 2026 randomized study of 164 law students compared no AI access, optional access, and optional access accompanied by approximately ten minutes of training. Training increased AI adoption from 26% to 41% and improved examination performance. Access without training did not improve performance and was associated with shorter answers relative to no access.

The study concerns law students and a specific issue-spotting examination; it does not prove that ten minutes of training is sufficient in professional practice. It demonstrates that access and enablement are separable interventions. Organizations that purchase tools without designing use cases and training users may understate both the technology’s potential and its risks.

Effective enablement should include

• approved and prohibited uses;
• account and data-handling requirements;
• the structure of a good task specification;
• the authoritative sources required for the matter;
• practice-specific examples and counterexamples;
• verification and escalation requirements;
• documentation and disclosure expectations; and
• criteria for deciding when not to use AI.

Prompt technique is only one component. Users also need to understand the workflow, source environment, and standard of review.

6. Professional judgment remains the control layer

ABA Formal Opinion 512 applies existing professional duties to generative AI rather than treating it as an exception. The opinion identifies duties involving competence, confidentiality, communication, candor, supervision, and reasonable fees. Specific obligations depend on circumstances and applicable jurisdiction, but the organizing principle is clear: using an AI system does not displace the lawyer’s responsibility for the representation.

NIST’s Generative AI Profile provides a complementary organizational perspective. It treats AI risk as a lifecycle problem involving governance, mapping, measurement, and management. For legal organizations, responsibility cannot sit solely with information technology or individual lawyers experimenting independently. The workflow must connect professional responsibility, information security, knowledge management, legal operations, and practice leadership.

Human review should be designed, not presumed

• the decision or output being reviewed;
• the identity and authority of the reviewer;
• the evidence the reviewer must inspect;
• the standard for acceptance;
• the conditions requiring escalation; and
• the record retained after approval.

Review intensity should rise with the consequence of error, novelty of the issue, opacity of the source path, and degree to which the output will affect a client, tribunal, counterparty, or regulated decision.

7. A workflow-level evaluation standard

Before deploying AI into a legal workflow, a firm or legal department should answer six questions.

1. Is the task sufficiently defined?

Identify the input, expected output, user, matter context, and point at which the work enters or leaves the legal process.

2. Are the necessary sources available and authoritative?

If the task requires current law, matter facts, executed agreements, or organizational policy, the system needs a reliable way to use those materials and the reviewer must be able to inspect them.

3. Can material errors be detected?

A workflow is a poor candidate for broad delegation when errors are difficult to identify, even by a qualified reviewer. Reviewability matters as much as theoretical capability.

4. Is the consequence of error proportionate to the control environment?

Low-consequence internal ideation and a filing submitted to a court should not share the same approval path.

5. Are users trained for this use case?

General AI literacy is not a substitute for practice-specific instruction, permitted-use guidance, and examples of acceptable work product.

6. Is value measured after the cost of review?

The relevant metric is not generation time. It is total cycle time and quality after instruction, source preparation, verification, correction, and escalation.

8. Implications for law firms and in-house legal teams

For law firms

AI creates an opportunity to reduce time required for selected tasks, but also exposes tension between efficiency and time-based revenue. Firms need a deliberate position on how gains affect pricing, staffing, associate development, quality assurance, and client communication. Treating AI only as an individual productivity tool leaves these operating questions unresolved.

For in-house legal teams

The value proposition is broader than reducing outside-counsel spend. AI may improve response time, increase consistency of recurring work, make institutional knowledge more usable, and allow legal professionals to spend more time on judgment and business partnership. Those benefits require cross-team workflow mapping, approved organizational accounts, and clear ownership after launch.

For both

The evidence favors controlled enablement over unrestricted experimentation or categorical prohibition. A viable program should give professionals approved tools and useful workflows while making source, review, data, and accountability requirements explicit.

Conclusion

The present evidence supports neither complacency nor indiscriminate adoption. Generative AI can create substantial legal-work productivity gains. Under some conditions, newer reasoning and retrieval-augmented systems can also improve quality. Yet results remain conditional, and public general-purpose models can be profoundly unreliable when treated as sources of legal authority.

The governing insight is operational rather than technological: value emerges from the designed human-AI system. Legal organizations should begin with defined work, authoritative sources, trained users, explicit review, and measured outcomes. The goal is not to maximize AI use. It is to produce legal work that is more efficient and capable without compromising accuracy, confidentiality, judgment, or professional responsibility.

References

1. American Bar Association Standing Committee on Ethics and Professional Responsibility. (2024). Formal Opinion 512: Generative Artificial Intelligence Tools. Official opinion PDF
2. Autio, C. et al. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. https://doi.org/10.6028/NIST.AI.600-1
3. Chen, B. M., & Bao, H. (2026). Training for Technology: Adoption and Productive Use of Generative AI in Legal Analysis. https://doi.org/10.48550/arXiv.2603.04982
4. Choi, J. H., Monahan, A. B., & Schwarcz, D. (2024). Lawyering in the age of artificial intelligence. Minnesota Law Review, 109, 147-216. https://doi.org/10.24926/265535.4225
5. Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024). Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis, 16(1), 64-93. https://doi.org/10.1093/jla/laae003
6. Nielsen, A., Skylaki, S., Norkute, M., & Stremitzer, A. (2024). Building a better lawyer: Experimental evidence that artificial intelligence can increase legal work efficiency. Journal of Empirical Legal Studies, 21(4), 979-1022. https://doi.org/10.1111/jels.12396
7. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192. https://doi.org/10.1126/science.adh2586
8. Schwarcz, D. et al. (2026). AI-powered lawyering: AI reasoning models, retrieval augmented generation, and the future of legal practice. Journal of Law & Empirical Analysis, 3(1), 220-250. https://doi.org/10.1177/2755323X261427048

This report is an evidence review for general informational purposes. It is not legal advice, does not create an attorney-client relationship, and should not be treated as a substitute for jurisdiction-specific professional judgment.