The real limit on AI agent autonomy is what it can't undo

Most teams decide how much autonomy to give an AI agent by how complicated its task looks. A summarization agent runs unattended. A payment agent gets a confirmation click. That rule of thumb feels reasonable. It measures the wrong thing.

A better measure exists: what happens if the agent gets it wrong. A complicated task with an easy undo is safer to automate than a simple task with no undo at all.

How much autonomy an AI agent actually needs

Anthropic published Measuring AI agent autonomy in practice in February 2026, studying how Claude is actually used across real deployments. Autonomy is not a fixed property of a model, the researchers found. It is "an emergent characteristic of a deployment, shaped by the model's behavior, the user's oversight strategy, and the product's design." The same model can run wide open in one product and stay on a short leash in another. The product's design decides which.

The same research put numbers on how agents are supervised in practice, across tool calls on Anthropic's public API. Eighty percent of tool calls appeared to come from agents with at least one kind of safeguard in place. Seventy-three percent appeared to have a human in the loop in some form. Only 0.8 percent of the actions agents took appeared to be irreversible. An example is sending an email to a customer that cannot be pulled back.

That last number is the useful one. The research does not say where teams put their oversight. It does show that actions that cannot be undone are a small share of what agents do. That makes a deliberate design move possible: concentrate oversight on that small share, and let the agent run everywhere else.

A decision rule for where to put the checkpoint

Before shipping autonomy for any task, ask one question: can this action be undone, at no real cost, before anyone downstream notices? If the answer is yes, a light guardrail is enough: a log, a rate limit, an alert if the agent repeats the same action an unusual number of times. If the answer is no, the checkpoint has to sit before the action executes. A summary of what already happened is a report. A report is not a checkpoint.

The National Institute of Standards and Technology frames a similar choice at the policy level. Its AI Risk Management Framework describes systems on a spectrum. Some systems "autonomously make decisions." Others "defer decision making to a human expert." Still others are "used by a human decision maker as an additional opinion." Reversibility is a practical test for choosing a point on that spectrum, action by action rather than for a system as a whole.

What this looks like across different products

Picture a finance team's AI agent that reconciles invoices against purchase orders all day without anyone watching. It flags mismatches, drafts vendor emails, and queues payments for release. Flagging and drafting are reversible. A wrong flag costs someone five minutes to dismiss. Releasing a payment is not reversible. The agent should own everything up to that line and stop there.

Imagine a support team's AI agent that drafts refund responses and can send anything under twenty-five dollars on its own. Above that threshold, approval has to happen before the refund goes out. A report a supervisor reads afterward does not count as a checkpoint. The dollar amount is a proxy for a simpler question: how much does it cost the business if the agent is wrong here? Can that cost be clawed back?

Say a marketing team's AI agent manages a client's social queue. It can draft and schedule posts into a review column on its own. It cannot publish to a client's live account without a named person clicking publish. A published post on someone else's brand is not something an internal log can undo.

A test to run this week

A workflow that fails the last two questions needs a checkpoint before it ships. Finding the gap after a customer notices it is the expensive way to learn the same lesson.

The metric this shows up in

When this is done well, escalation rate does not have to rise as more work goes to an agent. It can hold steady, because the checkpoints sit on the handful of actions that were always going to need a person. If escalation rate or the rate of overridden agent actions climbs alongside agent volume, the checkpoint is likely sitting in the wrong place. More often, the fix is moving it rather than adding more of them.

The approval moment itself is its own design problem once you know where it belongs. See the piece on why agentic products live or die at that step for what the moment itself needs to work. For what to show a user when the agent gets something wrong anyway, see the piece on designing for AI failure states.

FAQ

Does more autonomy always mean more risk

No. A highly autonomous agent that only ever touches reversible actions can carry less risk than a cautious one that still touches a single irreversible action with no checkpoint. Autonomy and risk move together by default. Designing around reversibility is what decouples them.

How do we measure whether an AI agent has too little oversight

Watch the override rate: how often a person reverses or corrects what the agent proposed. A very low override rate on high-stakes actions can mean the agent is excellent. It can also mean nobody is actually checking. Compare that rate against how often a reviewer changes complicated work a person authored in the same process.

What if reversibility itself is unclear

Treat the action as irreversible until proven otherwise. The cost of an unnecessary checkpoint is a moment of friction. The cost of a missing one is a mistake nobody can take back.

Sources

• Anthropic, Measuring AI agent autonomy in practice, February 2026
• NIST AI Risk Management Framework, Appendix C: AI Risk Management and Human-AI Interaction

If your team is about to hand an agent a workflow with real consequences, we help find where that checkpoint actually belongs before you ship it. Reach out and we can walk through your workflow together.