Essay 02

A Human Was in the Loop. Could They Change the Decision?

A reviewer can recognize a mistake while the decision continues unchanged. What would show that human judgment had a practical way to affect it?

First repository record ·

Research context

Methods phase complete · Three historical cases

The reimbursement scenario is hypothetical (it appears in the essay below, not in the manuscript).

Start with the guided study notebook

A reviewer can find the error and still be unable to stop the decision.

Consider a hypothetical reimbursement request. An AI system recommends rejection because a receipt is missing. The reviewer opens the file and finds the receipt you supplied. They have a reason to question the recommendation. You have a reason to expect that their discovery will matter.

Between those two expectations lies the problem. The reviewer might be able to pause the rejection, correct its basis and have the request reconsidered. They might also be able to leave a comment while the rejection proceeds. Both arrangements can contain a step called human review.

Trust, Autonomy, and Evidence asks what would let us tell the difference. Its subject is the opportunity for practical human control, and the evidence needed to assess whether that opportunity existed and took effect. A claim that someone reviewed a decision leaves much of that work ahead.

The room to reconsider

The receipt gives the reviewer something the recommendation alone cannot provide: a basis for questioning its stated premise. A summary that repeats “receipt missing” would keep the review inside the same incomplete account. Access to the underlying file opens the possibility of a different judgment.

That judgment still needs work. The receipt could belong to another request, or the expense could fall outside the policy. The reviewer has to test the apparent correction as carefully as the original recommendation. The question is what the available evidence supports.

A persuasive explanation can complicate that task. In experiments reported by Gagan Bansal and colleagues, explanations increased acceptance of AI recommendations whether those recommendations were correct or incorrect. They did not improve the combined performance of people and AI in the tasks studied. A reviewer therefore needs a way to check the answer’s grounds, including when its explanation is convincing. The effect of explanations in this hypothetical service would require testing.

Careful judgment also consumes time. Suppose the company rewards reviewers for closing requests quickly and treats a pause as a missed target. The interface offers permission to investigate; the working conditions attach a cost to using it. An assessment of practical control has to examine both. A button cannot tell us whether exercising the authority behind it is realistic.

The aim is proportionate scrutiny. Routine requests may need little investigation. Here, a specific contradiction justifies a pause: the company is preparing to reject a request for lacking a document that appears to be present.

Permission has to reach the action

Suppose the reviewer checks the receipt, records the concern and selects “hold.” The case screen now says “paused.” What has stopped?

A separate service might already be preparing the rejection message. The customer’s account might still be marked “rejected.” A hold on the reviewer’s screen establishes very little about those other steps until their response is examined.

This is a design question with a testable consequence. The organization must identify which actions the hold is meant to prevent, how the instruction reaches them and what shows that they stopped. When an instruction crosses a team or system boundary, authority has to remain effective across that boundary. Otherwise the reviewer can be responsible for a decision they lack the means to control.

Timing changes the remedy. Before the rejection is sent, the company may be able to prevent it. After delivery, a correction must reach the person who received it and any records that carry the mistake. The same instruction can arrive in time to change one consequence and too late to change another.

A successful hold also creates an obligation. Someone still owes you a decision. The organization needs a person authorized to resolve the dispute, an escalation route when they cannot, and a way to keep the request from disappearing into an indefinite pause.

What changed for the customer?

Now suppose an auditor finds the receipt and the reviewer’s hold request. The record ends there.

Several explanations remain possible. The hold worked. The rejection continued. Another process stopped it for an unrelated reason. The retained record cannot distinguish those outcomes. Missing evidence leaves the finding unresolved; it does not establish that the control failed.

This is why the research needs an explicit assessment method. A reassuring entry can survive even when the consequential part of the story is missing. The assessor needs to state which claim each record supports and what further evidence would settle the remaining question. Otherwise “review completed” can quietly become “control worked.”

A changed answer is an imperfect shortcut too. After examining the receipt, the reviewer could reasonably uphold the rejection because the expense is ineligible for another reason. Conversely, a decision could change through an automated correction without anyone exercising judgment. Counts of approvals, overrides or disagreements would leave those differences unexplained.

Practical control concerns whether relevant evidence could inform an authorized judgment and whether the process could respond. Establishing what happened in a particular case requires a record connecting those steps. Even then, a written account of a reviewer’s reasons cannot fully establish what they understood.

What the research can establish

The question becomes more demanding when the consequences cannot be repaired. In March 2003, a Patriot missile crew fired on a British Tornado aircraft after the system identified it as an incoming missile. Both people aboard were killed. The UK Ministry of Defence’s accident summary states that the crew complied with the self-defence rules then in force. It also identifies shortcomings in the information and training available to them.

A person authorized the action. Understanding the failure requires examining what the crew could know and what opportunity existed to question the identification. The fatal outcome alone cannot answer every question about those conditions. Nor can this historical case establish how a contemporary AI service will behave. Its relevance is the need to examine the substance of an opportunity to intervene.

Trust, Autonomy, and Evidence develops a procedure for that examination. It asks whether a person could obtain and understand relevant information, had authority and a realistic opportunity to intervene, acted, and affected what followed. The procedure makes the grounds for each judgment available for scrutiny, including where the evidence runs out.

The demonstration assessed three selected historical cases: two Patriot cases and the Soviet Oko early-warning case. One researcher made the assessments. The preserved historical baseline produced two Fail findings and one Unresolved finding. The author-approved October reassessment leaves all three Unresolved across 64 retained combinations of partial and indeterminate judgments. The historical and revised records remain available for comparison.

Those 64 combinations explore the consequences of retained judgment alternatives. They are not 64 independent cases. The evidentiary validity of those judgment alternatives remains unresolved. The result is conditional: the revised assessments establish neither successful control nor its absence. A justified failure of a necessary condition still produces Fail. Independent reviewer reliability, transfer to contemporary services and improvements in decision outcomes remain unestablished.

The revisions show why the method must expose its own assumptions. An earlier Oko assessment marked all six stages as supported. A later adjudication addressed a mismatch between those judgments and the requirement for direct, contemporaneous evidence. With the historical evidence fixed before reassessment, all six stages became partially supported. The correction narrowed the permitted claim. It did not establish that the reported historical actions never occurred.

Human participation needs examination on other grounds as well. In a meta-analysis of 106 experiments, Michelle Vaccaro, Abdullah Almaatouq and Thomas Malone found that human–AI combinations, on average, performed better than humans alone and worse than the better of the human or AI working alone. Results varied across tasks and study designs. Task performance and the ability to challenge an action are distinct objects of assessment. The study supplies a reason to examine what the combination achieves; it does not validate TAE.

The contribution is therefore bounded. TAE makes an assessment procedure and its disputed judgments inspectable. Further research must determine whether others can apply it consistently and whether its distinctions help evaluate control in present-day systems.

The choices behind oversight

For a leader deploying AI, the practical question begins with the decision people are being asked to oversee. What may the system do? Which uncertainty warrants intervention? Who can act before the consequences become difficult to reverse?

MIT CISR’s AI Decision Matrix organizes responsibilities according to ambiguity and risk. Ambiguity concerns how clearly the available information settles the answer; risk concerns the consequences of error. The framework considers who frames the problem, carries out actions and learns from outcomes, while retaining human accountability as the division of work changes.

In the reimbursement example, finding a receipt and authorizing a rejection carry different responsibilities. An organization can assign each activity to a person or system and still leave the connection between them untested. The receipt matters to the customer only if someone can get it considered before the decision, or use it to challenge and correct a decision already made.

The NIST AI Risk Management Framework addresses the organizational arrangements: GOVERN 2.1 calls for documented roles and clear communication, while GOVERN 3.2 addresses responsibilities for human–AI oversight. Those expectations help specify what an organization should arrange. Evidence from a particular decision is still needed to establish what the arrangement allowed and what actually occurred.

An engineering team could begin with a bounded test of a disputed decision. Introduce information that warrants reconsideration. Give the reviewer the authority the service claims to provide. Then trace whether the concern reaches each action that needs to stop, and whether the resulting decision reaches the person affected. Specify in advance what would count as success, failure or an unresolved result. That makes the exercise capable of challenging the claim.

The test should also leave room for evidence that the review itself is mistaken. A reviewer could attach the wrong receipt or misunderstand the policy. Their intervention needs a route to examination too. Giving a person authority does not make their judgment infallible.

There are costs to this approach. Holds delay legitimate payments; investigations consume limited expertise; detailed records can expose personal information. The appropriate response depends on the consequences of error, the time available and the evidence needed to resolve the dispute. A test under quiet, well-resourced conditions leaves open how the process behaves under ordinary workload or pressure to keep cases moving.

These are decisions an institution must make before relying on an assurance of control. Someone must own the unresolved hold, the failed handoff and the correction that never reached the customer. Each is a place where a promise of human review can stop short of the consequence it was meant to govern.

An answer someone can question

Return to the receipt. You need to know whether it was considered, whether the rejection was paused and what decision followed. You also need a route to challenge an explanation that remains wrong.

The company could ultimately uphold the rejection. It could approve the expense. Either answer deserves examination on its reasons and on the process that gave those reasons practical force.

The research asks what would justify trusting that process. Its demand is specific: follow a reason to reconsider through a person able to act, into the decision that affects someone. An institution promising human control should be able to show how a well-founded objection can change what happens next.

AI use & disclosure

Disclosure updated November 8, 2026

Codex (OpenAI) assisted with source discovery and inspection, proposed interpretations, manuscript drafting and substantive revision, formulas, figures, code and checks. Mark Julius Banasihan remains responsible for adopted claims and publication. Recorded author authorization does not mean every retrieval was personally reproduced or independently verified. Jev was used separately in synthetic experiments; all 1,079 original adjudications remain unresolved, and human time or quality benefits were not measured. The public records identify selected activities and limits, not a complete assistant transcript or independent peer review.

Codex (OpenAI) also assisted this website essay’s drafting, revision and source-record inspection. That editorial work does not establish independent human verification of every underlying source.

The November 8 editorial revision and guided notebooks received AI-assisted source checks, citation corrections and separate AI-assisted review. These checks are not a complete full-text audit of every reference or independent human validation. The notebooks distinguish preserved records, later explanations and missing historical material.

Author-review scope ↗

Program AI-use statement