Artificial intelligence can write our emails, summarize meetings, reconcile expenses, and increasingly take actions on our behalf.
Few people would object to an AI system automatically processing a routine $20 expense. But most of us would feel differently about letting that same system transfer $2 million, reset an executive’s credentials, or approve access to a sensitive environment without additional oversight.
That distinction sits at the heart of how organizations are thinking about human oversight of AI. In a human-in-the-loop (HITL) model, a person must review or approve an action before AI executes it. In a human-on-the-loop (HOTL) model, AI can execute while a person supervises the system and intervenes when something falls outside expected parameters.
Both approaches play an important role, depending on the level of risk. But they share an increasingly important vulnerability: they assume the human is actually there.
But how do we know the human in (or on) the loop is actually human?
As AI becomes better at imitating human appearance, voice, language, and behavior, simply designing a person into an oversight workflow is no longer enough. Organizations increasingly need to establish that a real person is present, that the person is who they claim to be, and that they have the authority to intervene.
The uncertainty of human oversight
AI provides speed and scale, and humans provide judgment, accountability, and a backstop when automation reaches its limits. The difference is when the human becomes involved.
HITL requires approval before an action proceeds, while HOTL allows the system to operate independently until intervention is necessary. Requiring someone to approve every calendar invitation or routine expense would eliminate much of AI’s intended efficiency. But as the consequences increase, so should human involvement and identity assurance.
We must ask ourselves: What level of human oversight does this action require, and how can we verify that the person responsible for that oversight is actually present?
That second question is becoming harder to answer. Historically, digital systems relied on signals that suggested human participation. Someone entered a password, responded to an email, appeared on camera, or clicked “approve.”
These signals weren’t infallible, but AI makes them even less meaningful. AI can generate messages, synthesize voices, manipulate faces, and increasingly operate software interfaces on people’s behalf. A system can therefore appear to have human oversight without proving that a live, authorized person is actually providing it.
Consider a self-driving vehicle. Depending on the system, a person might be expected to sit behind the wheel ready to take control, or a remote operator may supervise the vehicle and intervene when necessary. In either case, the safety model assumes that a human is available when the system needs one.
But what verifies that assumption? Is the driver alert and present? Is the remote operator actually monitoring the vehicle? Could software impersonate the signals that indicate someone is supervising? Simply assigning a human to the oversight model does not prove that the human is really there.
AI can imitate but cannot become human
Biometrics can take on a role here beyond traditional authentication. Liveness detection can help establish that distinction. Combined with biometric identity verification, it can help determine whether a live person is present and whether that person is the individual expected to approve or supervise an action.
The security objective therefore becomes more sophisticated than simply confirming that an action was approved. Organizations need confidence that a live person is present, that the individual has the authority to act, and, in a HOTL environment, that the human responsible for supervision is present and available to intervene when needed.
Organizations may increasingly need what we might call a “human identity checkpoint.” This is a deliberate point in an automated process where the system establishes sufficient confidence that a live, authorized person is either approving a consequential action or actively fulfilling the oversight role the system depends on.
This doesn’t mean adding biometric verification to every AI interaction (and adding more friction). Instead, identity assurance should rise alongside risk.
A routine action might happen autonomously. Moderate-risk activity could operate under a HOTL model, with a verified person supervising and intervening when necessary. Higher-risk actions could shift to HITL, requiring human review before execution. The most consequential decisions may require both verified human presence and explicit authorization.
In a HITL model, a human identity checkpoint might verify the person before an action proceeds. In a HOTL model, it may need to periodically establish that the expected person remains present, engaged and authorized to intervene. The greater the consequence, the stronger the assurance should become.
AI needs graduated autonomy
This points toward graduated autonomy as a more useful model for AI governance.
The debate around AI is often framed as a choice between giving machines autonomy and keeping humans firmly in control. In practice, different actions should carry different levels of autonomy, supervision, and identity assurance. That preserves the benefits of automation without treating every decision as if it carries the same level of risk.
Simply assigning human oversight doesn’t make a system secure. People may be distracted, unavailable, socially engineered, or operating through compromised accounts. In a HOTL environment, an even more basic risk arises when the human who is supposed to be supervising may not actually be supervising, or may not be there at all.
As AI becomes increasingly capable of impersonating people and activity, organizations will need to verify the human oversight layer itself.
Identity becomes a boundary for AI
For years, cybersecurity focused on determining whether the right user has access to the right resource. Autonomous AI raises the question of whether a human is performing an action, a machine is acting with human authority, or a machine is operating beyond the provided boundaries.
As machines get better at looking, sounding, and behaving like us, AI security depends not simply on putting humans in or on the loop, but on proving the human is real.
Ajay Amlani is president and CEO of Aware.