A Human Was in the Loop. Could They Change the Decision?
A person reviewed the answer. What matters next is whether they had the information, authority and means to change its consequences.
From the research of Mark Julius Banasihan
Essay draft · The reimbursement scenario is hypothetical.
Imagine receiving a message that a company has rejected your request for reimbursement. Its clear explanation cites a rule and assures you that a person reviewed the decision. Yet a document you supplied appears to have been overlooked. You ask for another look, expecting that someone who can understand the problem can also do something about it.
That expectation makes human control a practical question. A review should leave room for information that changes the answer, and for someone to act on it. The phrase “a human was in the loop” tells us that a person participated. It leaves open how much they could see, what they were allowed to change and whether their judgment reached the systems carrying out the decision.
Now go back to the reviewer’s desk, before the message is sent. An AI system has recommended rejecting the claim. Its summary looks complete, but an amendment in the customer’s file may change which rule applies. The reviewer has found a reason to pause. Whether that pause can protect the customer from an incorrect rejection depends on decisions made elsewhere: how the work is organized, who has authority and what happens when a person uses it.
This is a hypothetical example, not a reported incident. It helps make a larger question concrete. When an institution promises human oversight, what would allow someone outside it to establish that the promise had practical force?
The room to reconsider
The amendment may change the answer. It may turn out to be irrelevant. The reviewer needs to examine it before either conclusion is justified. A sensible process would make the original documents available alongside the AI summary and give someone with the relevant knowledge enough time to compare them. The review would also need a way to seek clarification when the record is incomplete.
In experiments reported by Gagan Bansal and colleagues, explanations increased acceptance of AI recommendations whether those recommendations were correct or incorrect. Explanations did not improve complementary team performance in the tasks studied. That finding gives a specific reason to examine what a reviewer can check behind a persuasive account; it does not establish how the hypothetical reimbursement service would behave.
Time is an organizational choice as well as a number on a clock. Imagine giving a reviewer a pause button while measuring their performance solely by how many claims they close. The button might work perfectly, yet using it could carry a professional cost. A manager who expects scrutiny has to account for that scrutiny in the workload and in the way the person’s work is judged. Otherwise, the institution has assigned a responsibility whose exercise it makes difficult.
None of this means every decision should be slow. Routine work can reasonably move quickly when the evidence is adequate and mistakes are readily corrected. In this example, the amendment supplies a particular reason for further attention: the company may be about to act on an outdated account of its obligation. The stakes and the uncertainty give the pause its purpose. The aim is to make careful judgment possible when it could matter.
Human judgment also needs a way to receive information from the person affected. The company’s file may contain a polished summary while the customer holds the missing detail. A route for supplying that detail, and a reviewer able to consider it, are part of the decision process. An appeal that only repeats the original explanation would leave the disputed premise untouched.
Permission has to reach the action
Suppose the reviewer examines the amendment and decides that the rejection should wait while its basis is checked. In our example, the company has authorized that person to place the claim on hold. They record the document, explain the concern and use the control that is meant to prevent release. The institution now has a human judgment to act on.
For a technology leader, this is where an apparently simple design question becomes an organizational one. Which action does the hold stop? The claims application might record the instruction while another service prepares the rejection email. A customer account might already have been updated. The word “paused” on one screen can describe only that part of the process unless the other systems have received and honored the same instruction.
The connections can be designed and examined. A hold might remove an item from the queue of decisions awaiting release. A separate acknowledgment might establish that the notification service received the change. A responsible team would need to know what happens if one service responds and another does not, who can investigate, and whether release remains blocked while the discrepancy is resolved. Those details determine the reach of the reviewer’s authority.
The reviewer should be able to understand that reach before relying on it. If stopping a notice requires escalation to another team, the route needs to work within the time available. If the notice has already been sent, the available action has changed: the company must now consider a correction. An instruction to stop and a correction after delivery address different circumstances. The record should make clear which one occurred.
What changed for the customer?
Suppose the case file shows the amendment, the reviewer’s reason for acting and the time the release queue accepted the hold. There is no record from the notification service. We can establish that the reviewer acted and that one part of the process responded. We still cannot establish whether the rejection reached the customer.
The missing record does not prove that the hold failed. The notice may have been stopped without an acknowledgment being preserved. It may also have gone out. A responsible account keeps those possibilities distinct and identifies the evidence needed to resolve them. For the customer, the difference is concrete: they may still be waiting for a fair decision, or they may already have received an answer the company had reason to reconsider.
A later reviewer would need records connected to this particular claim: what information was available, who issued the hold, when the relevant systems received it and what happened to the notice. More data is useful only when it helps answer the question. A large collection of logs that cannot connect an instruction to the affected decision may leave the central uncertainty intact.
Even a well-kept record has limits. It can omit pressure on the reviewer, a misunderstanding, or something that happened outside the recorded process. A timestamp establishes when an entry was made; by itself, it does not establish that the entry is true. Evidence of human control therefore needs examination of both the account and the circumstances it describes.
Reading the intervention
The reviewer pressed pause.
Did the rejection stop?
A reason to pause
An amendment may change the rule used to reject the claim.
Authority exercised
The reviewer records the amendment, the reason and an authorized hold.
A response recorded
The queue of decisions awaiting release acknowledges the hold.
The customer’s outcome?
The record does not establish whether the rejection notice was sent.
What the research can tell us
Trust, Autonomy, and Evidence develops a way to examine these connections in a specific decision. The Practical Human Control method asks whether a person could obtain and understand the relevant information, had authority and a realistic opportunity to intervene, exercised judgment, and could carry that judgment through to execution. The questions give an independent reader places to examine a claim that might otherwise rest on a role description or an approval record.
The historical demonstration is deliberately small: three selected cases, assessed by one researcher. Two concern the Patriot air-defence system; the third concerns the Soviet Oko early-warning system. In the reported assessment, both Patriot cases fail the complete event-control test despite evidence supporting intervention authority. Oko remains unresolved because the evidence only partly supports each required stage. These are findings under the method’s stated rules and source boundaries, not a measure of how often human control succeeds or fails.
The distinction between authority and its practical effect is useful to examine, but these historical systems differ materially from today’s learned AI models and agents. The study has not established that independent reviewers will consistently reach the same judgments across cases, or that the method works across contemporary settings. It also provides no evidence that using it improves safety or prevents harm. Those questions require further research. The present contribution is an inspectable procedure and a bounded demonstration of its application.
The wider evidence also cautions against assuming that human participation improves performance. In a meta-analysis of 106 experiments, Michelle Vaccaro, Abdullah Almaatouq and Thomas Malone found that human–AI combinations, on average, improved on humans alone but underperformed the better of the human or AI working alone. Results varied across tasks and study designs. This concerns measured task performance, a different question from whether people retained authority to challenge an action. It supplies context for scrutiny, not validation of TAE.
The research also exposes a difficulty in its own earlier account. An initial assessment treated the six Oko stages as supported. A later reassessment used a stricter evidence requirement, including direct and contemporaneous support, and changed all six to partially supported while preserving the earlier assessment. The historical packet stayed the same. The correction history records a change in what the evidence justified saying, without claiming that the historical action had been disproved.
That distinction matters beyond this study. An institution may discover that an assurance was stronger than its records could support. It should then revise the assurance and preserve the reason. Correcting the description is one obligation; finding out whether the underlying decision still needs correction is another. A more accurate report cannot by itself change the consequences already experienced by a customer.
The choices behind oversight
For leaders introducing AI into consequential work, the inquiry begins before a disputed decision arrives. Someone has to decide which concerns require a pause, who can authorize one, and how much time remains before an action becomes difficult to reverse. Those choices belong with the design of the service, because they determine what its promise of human review can mean.
The NIST AI Risk Management Framework addresses this organizational responsibility: GOVERN 2.1 calls for documented roles and communication, and GOVERN 3.2 for defined responsibilities in human–AI oversight. These are governance expectations. Evidence that a particular intervention reached execution must still come from the system and decision being examined.
One practical starting point would be to follow a disputed decision through the proposed process. Give the reviewer information that could change the answer. Examine whether they can understand it, obtain help, act within their authority and establish what happened next. If a hold crosses several systems, examine those handoffs. Such an exercise would supply evidence about the conditions tested; it would still leave questions about how the process behaves under ordinary workload, conflicting incentives or an unfamiliar problem.
There are real trade-offs. A pause can delay a legitimate payment or service. A review requirement can consume scarce expertise. A record can contain sensitive information that should not be available to everyone. The institution needs to define which concerns justify delay, how long a hold can remain unresolved and which evidence a reviewer needs to see. Preserving practical control calls for decisions about those costs, with responsibility for revisiting them when the circumstances change.
Nor must every review overturn the machine’s recommendation. A reviewer may examine the amendment and reasonably conclude that the original rule still applies. Human control includes the ability to reach that conclusion on the relevant evidence, through a process that could have acted differently had the evidence warranted it. Counting overrides alone would miss that distinction. The inquiry concerns the opportunity for judgment and the evidence of its exercise.
An answer someone can question
Return to the customer asking the company to reconsider. They should be able to understand which part of the decision is disputed, provide relevant information and reach someone with authority to act on it. The company should be able to explain what was reviewed and what followed. If the rejection was sent in error, the question becomes whether the correction reaches the customer and any other record that continues to carry the error.
That expectation extends beyond the quality of the AI’s answer. It concerns the institution’s capacity to remain responsive after an answer has been produced. Better models may reduce some mistakes. The people operating a service still need to decide what happens when the evidence changes, a reasonable objection arises or an action has already gone too far to stop.
An assurance of human review should open a decision to examination. The reviewer needs room to judge and a means to act. The person affected needs a route to challenge and, where warranted, correction. When the institution cannot yet establish what happened, its account should identify the gap and who can investigate it. The next answer should tell the customer what changed, what still needs attention and where their challenge can go.