TL;DR: Most teams want to move from workflows that call an agent to agents that run the work, but they still need to audit and check every outcome. The way through is not a leap, it is a ladder. Start with the lowest risk work, make every decision auditable from day one, and let autonomy be earned with measured results, not granted because the demo went well. Build the evals, traces and guardrails first, then demote your human gates from blocking checks to sampled audits as the data allows.
Let me start with a workflow I will call the Tuesday queue. It is a composite, built from a pattern I keep running into, not a single client.
A logistics company receives a few hundred carrier invoices a day by email. A Logic App picks each one up, calls an agent to extract the lines and match them to the purchase order, and then does what every sensible architect does. It stops and waits for a person to approve the result. The workflow is the brain. The agent is a very clever function call. The human is the safety net.
In month one it is brilliant. The agent is right nearly every time and the reviewers catch the odd miss. By month three the approval rate is 99.8 percent, the average review takes eleven seconds, and nobody is really reading the invoices any more. The gate is still there. It has just stopped checking anything. Then a carrier resubmits an invoice under a new reference number, the matching rules written two years ago do not recognise it, a tired reviewer clicks approve, and the same bill is paid twice.
Meanwhile the cases that need real judgement sit in the same queue behind hundreds of easy ones, and finance wants to know why the AI project saved so little time.
That is the uncomfortable truth about the deterministic gate. It feels like control. Quite often it is only the feeling of control. So sooner or later someone senior asks the obvious question: why are we not just letting the agent run the whole thing? Heal itself, audit itself, report on itself, learn from its own mistakes.
Fair question. Slightly terrifying answer. The move is not about making the AI smarter. It is about moving authority. Today the workflow owns the control flow. In the autonomous version the agent owns it and the platform becomes the guardrail. Almost all of the real engineering lives in that gap.
I have written before about my own rule that AI proposes and humans dispose. This post is the awkward sequel. How do you stop disposing of every single action without doing something reckless?
The Problem We Are Actually Solving
Most teams want to move to agentic workflows, and the reasons are sensible. An agent handles the long tail of cases nobody wrote a rule for. Work stops queuing behind people. Volumes can grow without hiring in step, and the board has read the same headlines you have.
The blocker is just as sensible. The moment the agent acts without a person checking each result, you still have to answer the questions that never go away. What did it do? Why? Was it right? Who is accountable if it was wrong? Auditors, regulators, customers and your own finance team will ask, and “the model seemed fine” is not an answer. That is why so many pilots stall at the first rung. The people who want the speed and the people who own the risk are both right.
So the problem is not making agents more capable. They are capable enough. The problem is how to get the speed and coverage of an autonomous agent while keeping the ability to audit and check what it did, without betting the business on day one.
The answer I will argue for has three parts. Start with the lowest risk work, where mistakes are cheap and reversible. Make every decision auditable from the first day. And let the agent earn more freedom, one class of action at a time, from measured results.
What success looks like: you can say how often the agent is wrong, what it is allowed to do on its own, and exactly how it loses that permission. By the end of this post you will have a ladder to plan with, the control plane that makes it safe, and a five step path with an exit test at every step.
Why Take the Gate Off the Critical Path at All?
Gates are not bad. A blocking check on every run is just a surprisingly poor way to buy assurance.
- A deterministic gate only catches the failures you predicted. The reason you reached for an agent is the long tail nobody wrote a rule for, and that is exactly where the gate is blind.
- Human gates decay. When a reviewer approves almost everything, the gate quietly turns into a rubber stamp, and you carry the cost of a queue without the benefit of a check.
- Gates make the hard cases wait. The five percent that need judgement sit behind the ninety-five that do not.
- Branches multiply. Every new exception becomes another path in the workflow. Deterministic orchestration turns into a maze long before it covers the long tail.
The alternative is not no control. It is control that scales: policy limits on every action, an independent audit on a measured sample, and people reading the cases where something actually disagreed. More assurance per hour of human attention. The rest of this post is about how to earn that, without the part where the same invoice gets paid twice.
The Ladder, Not the Leap
The organisations that get this right treat autonomy like a credit rating. You start with a small limit, prove you are good for it, and the limit goes up. In practice that means starting with the lowest risk workflow and action class, not the most impressive one. Each rung is earned with data from the rung below.
| Rung | Who owns control flow | How outcomes are verified |
|---|---|---|
| 1. Agent as a step | The workflow | Hard gate on every run, often a human |
| 2. Bounded autonomy | The workflow, with the agent choosing tools inside a step | Schemas, assertions, a second model as judge |
| 3. Agent plans, human approves | The agent proposes, the engine executes | Plan review and plan accuracy tracking |
| 4. Agent-owned execution | The agent | Policy gate plus sampled independent audit |
| 5. Self-improving | The agent, including its own prompts and runbooks | Regression suite before any change is promoted |
Here is the bit that does not make it onto the conference slide. Not every workflow should reach rung five. Some should stop at rung two and stay there forever. That is not a failure of ambition. That is architecture.
What “Self” Actually Means
Self-healing, self-auditing, self-reporting, self-learning. They sound like features. They are design constraints you either build in or discover the hard way. Self-reporting is the easy one: build the trace below from day one and reporting is just a query.
Self-healing is failure classification plus idempotency
Retrying everything is not healing, it is hoping. Classify the failure first, because each class needs a different response.
- Transient (throttling, timeouts): retry with backoff and jitter.
- Bad input (malformed document, missing field): repair or route to a human, never retry.
- Tool failure (API changed, credential expired): stop and raise an incident. Do not let the agent improvise.
- Wrong approach (the plan does not work): replan, with a hard cap on attempts and spend.
Then comes the part people skip. Every side effecting step needs an idempotency key and a checkpoint, so a retry or a replan cannot repeat work that already happened. Nothing ruins a Tuesday like a self-healing agent that cheerfully healed the same payment three times.
Self-auditing means an independent auditor
An agent marking its own homework mostly agrees with itself. Shocking, I know. A real audit needs: Different context, so it does not inherit the actor’s reasoning. A different model where you can manage it, so errors are less correlated. A different identity with read only access. And an append-only record to read, not whatever the actor chooses to say.
Self-learning is the one that keeps me up at night
The safe versions are a memory of past incidents and their fixes, curated example libraries, and prompt or runbook changes that must pass a regression set before promotion. The unsafe version is a system quietly rewriting its own behaviour in production. If you have read my post on why agents give different answers every time, you already know why that is a bad idea.
So how do you make learning safe? Treat every proposed change, whether a new prompt, a tool description or a runbook step, like a code change. It gets a version, then a replay against the golden set with per class thresholds, then a shadow run alongside production where its decisions are logged but not acted on, then a small canary share of traffic with the ratchet reset to full audit for that share. Only then does it promote. That much ceremony is deliberate, because changes, not steady state, are where regressions arrive.
The Control Plane You Need Around the Agent
When the agent owns the control flow, something else has to own the rules. There are six components, and the model is only one of them. For each, here is why it exists and how I would build it on Azure.
| Component | Why it exists | How on Azure |
|---|---|---|
| Execution layer | Retries, timeouts and checkpoints must be deterministic, or healing turns into guessing | Logic Apps or Durable Functions, with an idempotency key on every side effect |
| Agent runtime | Does the thinking and nothing else | Azure OpenAI or the Foundry Agent Service, with its own user-assigned managed identity and a fixed tool list |
| Policy gate | Rules the model cannot be talked out of | Plain workflow conditions or code reading a versioned classification table |
| Evidence store | An audit needs a record the actor cannot quietly rewrite | Immutable blob storage for the record, Table Storage or Cosmos DB for fast queries |
| Auditor | Independent checking only works if errors are not shared with the actor | A different model deployment under a different managed identity, read only on the evidence |
| Ratchet and kill switch | Turns evidence into permission, and takes permission back | A state row read at the start of every run, with role removal as the hard stop |
Two of those choices deserve a sentence each. Every agent gets its own managed identity because it limits the blast radius, lets you attribute every action to one agent, and gives you the strongest off switch you have: remove the role assignment and the agent cannot reach its tools, even if your own logic is wrong. And be careful with the evidence store. Immutable blob storage with a retention policy is genuinely write-once. Table Storage and Cosmos DB are not, so enforce it by convention and permission: the actor only ever inserts, never updates, and a scheduled export lands in immutable storage. Do not treat Logic Apps run history as your audit log either. It is operational telemetry with limited retention.
Trace the decision, not just the call
Logs tell you what ran. Traces tell you what the agent decided and why. Each agent step should emit a record like this. The OpenTelemetry conventions for generative AI are a sensible vocabulary if you also want it in Azure Monitor and Application Insights.
{
"runId": "2026-10-19T09:14:07Z-8f3a",
"step": 4,
"agent": "claims-triage-v12",
"model": "gpt-4.1",
"promptVersion": "triage-2026-10-02",
"inputHash": "sha256:9c1f...",
"decision": "approve_payment",
"confidence": 0.82,
"rationale": "Invoice matches PO 44187, amount within tolerance",
"actionClass": "irreversible",
"policyResult": "escalate",
"auditSelected": true,
"tokens": { "in": 1840, "out": 212 },
"costUsd": 0.011
}Every field answers a question you will ask during an incident: what did it see (the input hash), what did it think (decision, confidence, rationale), which version did it (model and prompt), was it checked (policy result and audit selection), and what did it cost.
Pin the model and prompt version in every record. When behaviour changes, “what changed” should be a query, not an investigation.
Action Classification and the Policy Gate
Autonomy gets granted per class of action, never globally. Drafting an email and moving money are not the same kind of verb. The classification needs to be explicit, versioned and boring.
{
"send_internal_summary": { "reversible": true, "maxValue": 0, "mode": "autonomous" },
"update_crm_record": { "reversible": true, "maxValue": 0, "mode": "autonomous" },
"approve_refund": { "reversible": false, "maxValue": 250, "mode": "sampled" },
"approve_payment": { "reversible": false, "maxValue": 5000, "mode": "gated" },
"modify_supplier_bank": { "reversible": false, "maxValue": 0, "mode": "blocked" }
}Why it lives outside the model. A model can be persuaded and a lookup table cannot. If the rule exists only in the prompt, a creative enough input, such as an invoice with instructions hidden in it, can talk the agent out of it. The gate treats the agent’s output as a proposal and nothing more.
How it runs. After the agent responds, parse the output against a typed schema so a malformed answer fails immediately. Look up the proposed action in the classification table. Then apply three rules that have nothing to do with the model’s opinion of itself. Irreversible actions are always audited. Anything below a confidence floor is audited. Anything over its value limit escalates to a person, which in a Logic App is an approval action in Teams or email. A blocked class terminates the run. The agent can ask for more authority all day long. It never gets to grant it.
The Eval Harness Is the Real Product
If you take one thing from this post, take this. The eval harness is what lets you trust change, and it gates every kind of learning. Build it from real production cases, not synthetic examples someone wrote on a Friday afternoon.
- Golden set. Real inputs with human-confirmed outcomes, including every incident you have ever had. Add to it after every failure.
- Replay. Run every prompt, model or tool change against it before production, and compare to the baseline.
- Calibrated judges. If a model grades outputs, measure its agreement with your human labels first.
- Per class thresholds. A drop on the irreversible class blocks release even if the average improves.
How to build the golden set. Sample from run history by action class, not at random, and deliberately oversample escalations, failures and disagreements, because the boring middle teaches you very little. Have two people label the irreversible cases and settle any disagreement between them. Hold back a fifth that you never tune against, or you will overfit your own tests. Start with a few hundred and add to it after every incident.
The Ratchet, and a Statistics Problem Nobody Mentions
The autonomy ratchet is the controller that turns measured results into permission. The principle is slow to trust, quick to distrust. An agent starts with every run audited. As clean audits accumulate, the audit sample rate steps down, for example halving from 100 percent through 50, 25 and 12 down to a floor of 5. Disagreements push it back up fast. The state is small: a sample rate, a window of audits, a count of disagreements, and a kill switch flag.
Why a floor, and why halving. The sample rate never reaches zero, because drift from new input types or a model update only shows up if you keep looking. Halving makes each step a small, reversible change in assurance instead of a cliff. How the state is held. One row per agent and action class: mode, sample rate, audit count, disagreement count and the kill switch flag. The run reads it first and updates it after each audited run. With trigger concurrency set to one there are no write races. At higher volume use ETag conditional updates and retry on conflict.
if audits < minAudits -> hold
else if disagreementRate > demoteAbove (say 5 percent) -> demote
else if audits >= windowAudits and
disagreementRate <= promoteBelow (say 2 percent) -> promote
else -> hold
reset the counters on any decision, or when the window fillsTwo details matter. Promotion needs a bigger window than demotion, or one lucky run of clean audits can re-promote an agent mid-incident. And the window tumbles and resets on every decision, because cumulative counters dilute a fresh problem into a long history of good behaviour.
Now the statistics. Suppose an agent passes 20 audits with zero disagreements. The rule of three says that with zero failures in n trials, the 95 percent upper bound on the true failure rate is roughly 3 divided by n. After 20 clean audits that bound is 15 percent. After 50 it is 6 percent. To claim below 2 percent with any confidence you need about 150 clean audits.
And sampling stretches it further. At a 5 percent sample rate, collecting 50 audits takes about 1,000 runs. That is not a reason to avoid the ratchet. It is a reason to size windows from the cost of a wrong outcome. My demo thresholds are deliberately quick so you can watch them move on stage. In production, an irreversible class needs a far longer, stricter window.
The kill switch. The flag in the state row is read first in every run. When it is set, the run terminates immediately with a 503 and nothing downstream executes. That stops new work, not runs already in flight, which is why you pair it with the harder stop of removing the agent’s role assignment.
A Migration Path for an Existing Logic App
Everything above is the destination. Here is the route, as five steps you can run against a workflow you already have. Each has a why, a how and an exit test, and you do not start the next until the test passes. If it fails, you stay put. That is the whole discipline.
Step 1: Add evidence without changing behaviour
Why: you cannot audit, debug or improve what you cannot see, and adding a witness changes nothing, so it carries no risk. How: give the Logic App a user-assigned managed identity, add a step after the agent call that writes the trace record to Table Storage or Cosmos DB with an HTTP action, and schedule an export to immutable blob storage for the tamper resistant copy.
Exit test: for a full week, every run has exactly one evidence record, and you can answer “what did the agent decide on this case and why” with a query.
Step 2: Build the golden set and baseline the current agent
Why: without a baseline you cannot tell whether any later change helped, and this same set will gate learning in the end. How: export a few hundred runs sampled as described above, label them, and replay them through the current prompt and model from a test harness. A second Logic App or a short script both work. Record the pass rate per action class and per input type.
Exit test: a repeatable replay produces a baseline, and any judge model agrees with your human labels often enough to trust.
Step 3: Classify actions and put the policy gate in front
Why: it moves the rules out of the model, where inputs can argue with them, and a typed schema means a bad answer fails at the parse step and not three actions later. How: add a Parse JSON action with a strict schema, store the classification table in Table Storage or App Configuration, branch with a Switch on the class and mode, use an approval action for escalations and a Terminate for blocked actions. The model proposes, the gate decides.
Exit test: every action the workflow can take has a class, and a deliberately bad test case (wrong amount, unknown action) is blocked or escalated every time.
Step 4: Add the independent auditor and audit everything
Why: you need the real error rate before you rely on it, and a second model’s disagreements are the cheapest labelled data you will ever get. How: add an HTTP action to a different Azure OpenAI deployment, using a second managed identity that holds only the Cognitive Services OpenAI User role and read access to the evidence. Set the temperature to zero, give it the input, the decision and the rationale, and require a JSON verdict of agree or disagree with a reason. For now the sample rate is 100 percent. The decision to audit is one expression, which makes it easy to turn into a dial later:
@or(
or(equals(body('Parse_State')?['mode'], 'gated'),
less(rand(0, 100), int(body('Parse_State')?['sampleRate']))),
or(equals(body('Parse_Actor')?['irreversible'], true),
less(float(body('Parse_Actor')?['confidence']), 0.7))
)Read it as: audit if the workflow is still gated, or the run falls inside the sample, or the action is irreversible, or confidence is low. Property names are illustrative, so match them to your own schema.
Exit test: the disagreement rate is measured, you have read a sample of the disagreements yourself, and you know whether the auditor or the actor was wrong in each.
Step 5: Turn on the ratchet for one action class
Why: it lets autonomy grow only from evidence, on the cheapest class first, so early mistakes cost little. How: after each audited run, compute the disagreement rate for the window, apply the decision rules, halve or double the sample rate, reset the counters, and write the state row back. Put the kill switch check at the top and rehearse it. Leave every irreversible class gated until you have far more evidence than the first class needed.
Exit test: you have watched the ratchet both promote and demote in a controlled test, the kill switch stops a live run, and you have rolled back a change using the golden set at least once.
Notice what that sequence never does. It never asks anyone to trust the agent before the evidence exists, and at every point you can stop with a safer workflow than the one you started with. I built a Logic App that implements the audit, ratchet and kill switch from steps four and five, and I will walk through it in a hands-on follow-up post.
Taking It Across Industries
Industries differ in two things: what a wrong outcome costs, and how reversible the action is. So the rung is set per workflow, not per industry. Build a horizontal core (traces, evals, classification, ratchet, kill switch) and add thin vertical packs of policies, domain evals and regulatory mappings.
- Finance: Reconciliation, reporting and KYC preparation can reach rung four. Payments stay gated, and in the EU DORA shapes what your evidence store must hold.
- Insurance: Claims triage and document extraction can go a long way. Denials and pricing need explainability and a human, and under the EU AI Act life and health pricing counts as high risk.
- Transport and logistics: Exception handling, re-routing and customs documents suit autonomy well: high volume, mostly reversible. Safety related actions are the limit.
- Manufacturing: Planning and quality analysis can be highly autonomous. Anything touching OT or control systems is a hard boundary: the agent recommends, it does not act.
A new industry should mean a new policy pack and eval set, not a rebuilt platform, and each one earns its own error rate.
Demote the Gates, Don’t Remove Them
The trap is ripping out the checks all at once because the pilot went well. Keep the gates, but demote them. A blocking check on every run becomes a sampled audit on a slice of runs, and only when the measured disagreement rate has earned it. The gate becomes a dial, and the data decides where it sits.
An existing Microsoft estate is an advantage here. Logic Apps stays as the durable execution and retry layer underneath an autonomous model. It just stops being the brain, and identity, policy and monitoring already live on the same platform.
Back to the Tuesday queue. The goal was never to remove people. After the climb, reviewers stop approving hundreds of invoices a day and start reading the handful where the auditor disagreed, plus every irreversible payment the policy gate escalated. A duplicate payment hits a deterministic check on the way out, because money never moves on the agent’s word alone. Fewer clicks, more attention, and evidence behind every decision. That is a better gate, not a missing one.
So here is where I would start. Take one workflow and ask two questions. What does a wrong outcome cost, and can it be undone? The answers tell you how far up the ladder that workflow should ever go. Then work through the five steps above, in order, and earn the first sample rate reduction with data. Then do it again.
Earn the autonomy. Then go build something.
If you are working out where your own workflows sit on this ladder, or you have a story about an agent that got ahead of its guardrails, I would love to hear it. Drop a comment below.