“Human in the loop” is often used to reassure stakeholders that an AI system remains supervised. But a person somewhere near an output is not the same as an effective control.
A phrase is not a workflow
Legal organizations already understand that review is contextual. A partner reviewing a dispositive brief, a contracts manager checking extracted renewal dates, and a general counsel reading an internal brainstorming note are performing different controls. They require different authority, evidence, time, and documentation.
AI does not remove that difference. It makes the design choice more important because fluent output can conceal weak sources, false premises, omitted issues, or fabricated authorities. A system may produce text that looks complete before the underlying proposition has been verified.
The control is not the presence of a human. The control is the work the human is required and able to perform.
Professional duties continue to apply when lawyers use generative AI. Organizational risk guidance likewise treats governance, mapping, measurement, and management as lifecycle responsibilities. The practical task is to translate those principles into a review path someone can actually follow.
Six elements of designed human review
1. The output or decision
Name what is being reviewed. “AI work product” is too broad. A citation list, factual chronology, clause extraction, first draft, risk classification, and strategic recommendation require different checks.
2. The reviewer and authority
Identify the person or role that can accept, reject, revise, or escalate the output. Expertise and authority are separate: a person may know how to inspect the work but lack authority to approve the legal decision.
3. The required evidence
Define what the reviewer must open or compare. Citations require the actual authority. Factual synthesis requires the record. Contract extraction requires the executed document. A strategic recommendation requires the assumptions, alternatives, and matter context.
4. The acceptance standard
Describe what “good enough” means. Accuracy, completeness, currency, jurisdiction, internal consistency, tone, defined terms, and client objective may all matter. Without a standard, reviewers substitute individual intuition.
5. The escalation path
State which conditions stop the workflow or require additional review: unavailable sources, high-consequence errors, novel issues, conflicting authority, sensitive data, or a reviewer who cannot verify a material proposition.
6. The retained record
Determine what should remain after approval: source references, versions, reviewer identity, exception notes, or the final accepted output. The right record depends on the matter and policy, but it should be a deliberate choice.
Review intensity should follow consequence and detectability
Not every task needs the same control. The intensity should rise when the consequence of error is high, the issue is novel, the source path is opaque, or a qualified reviewer would struggle to detect a mistake.
| Output | Minimum review design |
|---|---|
| Internal ideation | Confirm that no unverified proposition becomes advice, record, or external representation. |
| Factual chronology | Trace every material event to an approved record and resolve conflicts. |
| Contract extraction | Compare extracted terms against the executed agreement and defined fields. |
| Legal proposition | Verify current primary authority, jurisdiction, text, and subsequent treatment. |
| Strategic recommendation | Require accountable legal judgment, explicit assumptions, alternatives, and escalation. |
Reviewability should influence task selection. A workflow is a poor candidate for broad delegation when material errors are difficult to detect even for a qualified professional.
The record turns review into organizational capability
A designed review path does more than reduce immediate risk. It produces evidence about failure modes, recurring corrections, source gaps, user needs, and opportunities for better configuration. That record makes future improvement possible.
This is the difference between individual vigilance and an operating control. Individual vigilance depends on memory and effort. An operating control defines the work, gives the reviewer what they need, and leaves enough evidence for the organization to learn.
Further reading
- Flintwood Research Report 001: Generative AI and Legal Work
- National Institute of Standards and Technology. (2024). Generative Artificial Intelligence Profile. https://doi.org/10.6028/NIST.AI.600-1
- American Bar Association Standing Committee on Ethics and Professional Responsibility. (2024). Formal Opinion 512: Generative Artificial Intelligence Tools.