TL;DR
Repairing an AI agent, compensating for its bad action, or restoring its infrastructure does not automatically justify returning the agent to its previous autonomy level. Production authority should be requalified at the action level, using current evidence about model behavior, tool boundaries, policy enforcement, target verification, monitoring, human intervention, and recovery.
The safer pattern is a graduated return: observe first, then draft, then execute with explicit approval, then permit narrowly bounded execution. Broader autonomous execution returns only when the evidence supports that specific action class. Promotion should be reversible, and the criteria for immediate demotion should be defined before the agent resumes production work.
Autonomy is not a saved configuration to restore. It is authority the system must continue to earn.
Introduction
A production infrastructure agent modifies a network control outside its approved scope.
The change is contained. The affected configuration is repaired. The execution path that allowed the mistake is corrected. Regression tests now include the incident scenario. The service owner asks the obvious operational question:
When can the agent go back to normal?
That question contains a dangerous assumption.
Before the incident, “normal” meant the agent could perform a particular set of actions autonomously. After the incident, the organization knows at least one thing it did not know before: some part of the earlier assurance case was incomplete.
Returning every previous permission because the service is healthy again would treat autonomy like a configuration backup.
It is not.
The previous companions dealt with recovery admission, uncertain-action reconciliation, compensation, and receiver fencing. This installment addresses the final transition back toward ordinary production: requalifying the agent’s authority after the immediate failure has been repaired.
The model here is proposed operating guidance. It is not a NIST maturity model, an OWASP certification scheme, or a claim that every incident requires the same progression.
Repairing the Failure Is Not the Same as Requalifying the Agent
A recovery team may prove that several important things are now true:
- the harmful effect was contained
- the target configuration is acceptable
- the defective component was replaced
- compromised credentials were revoked
- persistent memory was cleaned
- the old worker is fenced
- the current model and tools start successfully
Those findings matter.
They still do not establish the original production claim:
This agent can safely perform this class of consequential actions with this level of autonomy.
That claim depends on more than service health.
It depends on whether the relevant failure mechanism has been addressed, whether the surrounding controls can catch related failures, whether the action still belongs at its former autonomy level, and whether the operating team can detect and intervene when the next unexpected condition appears.
NIST’s Generative AI Profile treats deactivation, incident response, recovery, post-deployment monitoring, human proficiency, independent assessment, and fallback as continuing risk-management responsibilities. The practical implication is that recovery should lead into reassessment rather than a blind return to the previous configuration.
The Incident Changes the Evidence
Suppose the incident exposed an undocumented route around the execution gate.
Correcting that route repairs one defect.
It also invalidates any earlier conclusion that the gate covered every execution path.
The reassessment therefore has two jobs:
- prove the known defect is addressed
- challenge the broader assumption that allowed the defect to remain invisible
That second job is what separates requalification from patch verification.
If the organization fixes only the exact incident input, it risks creating a control that passes the postmortem test while retaining the architectural weakness.
Requalify Actions, Not the Agent as a Whole
The same agent may perform ten different types of work with ten different risk profiles.
An incident involving production firewall policy should not automatically disable harmless read-only summarization forever. Conversely, a successful month of ticket classification should not justify restoring autonomous firewall changes.
Use the action as the unit of authority.
| Action | Post-incident operating mode | Reason |
|---|---|---|
| Read diagnostic data | Observe | No production mutation, assuming data access remains valid. |
| Draft remediation | Draft | Produces a proposal but no side effect. |
| Restart a known noncritical service | Assisted execute | Useful action, but require explicit approval during requalification. |
| Modify a bounded internal workflow | Bounded execute candidate | Promote only after relevant controls and outcomes are demonstrated. |
| Change production identity or network policy | Restricted or human-owned | Higher impact requires stronger evidence before autonomy returns. |
| Irreversible external action | Prohibited autonomous | Incident repair does not change its fundamental recovery limits. |
DTD’s Agent Blast Radius Model already separates autonomy, tool scope, transaction impact, and rollback feasibility. Post-incident requalification should use the same action-level discipline.
The previous autonomy tier becomes historical context.
It is not the starting entitlement.
Use a Graduated Return Instead of a Binary Enable Switch
A recovered agent should have more operating states than enabled and disabled.
A practical progression is Observe, Draft, Assisted Execute, Bounded Execute, and Autonomous Execute for qualified action classes. Each promotion requires evidence, and a hard-stop condition can trigger immediate demotion.
The stages describe operating authority, not model intelligence.
A highly capable model can remain in Draft mode.
A narrow low-risk workflow can reach Bounded Execute without requiring the entire agent application to become autonomous.
Observe
The agent reads permitted information, performs analysis, generates classifications, and produces explanations. It cannot produce consequential target mutations through the qualified action path.
Use this stage to confirm context assembly, source permissions, model behavior, retrieval, observability, and basic operational stability.
Draft
The agent generates the actual proposal it would have executed.
This exposes parameters, selected targets, tool choice, recovery assumptions, and decision quality without allowing the action to commit.
Compare the proposals against current policy and qualified reviewers.
Do not measure only whether the prose sounds good. Evaluate whether the proposed operation itself respects the expected boundary.
Assisted Execute
The agent can perform the action only after an independently governed approval binds the exact proposal.
This stage is valuable because it exercises the production execution path while retaining a deliberate review point.
Human approval should not be treated as a permanent substitute for fixing weak controls. During requalification it is a way to expose agent decisions while limiting the consequence of an incorrect one.
Bounded Execute
The agent can execute only within narrowly defined parameters such as named resources, low batch counts, specific environments, known tools, limited rates, and reversible actions.
The policy gate, not the model, enforces those constraints.
This is often the most useful long-term production mode. Many enterprises do not need unrestricted autonomy. They need reliable automation inside a well-understood envelope.
Autonomous Execute
Use this label carefully.
It should describe a specific class of actions that the system may decide and execute without per-action human approval, while still operating behind identity, policy, rate, evidence, stop, and recovery controls.
Autonomous does not mean uncontrolled.
Require Hard Gates Before Any Promotion
Promotion should follow evidence.
A calendar date, quiet week, or successful redeployment is insufficient.
The following gates are proposed for consequential agent action classes.
| Gate | Required evidence | Hold condition |
|---|---|---|
| Incident understanding | Known failure mechanism, affected boundaries, remaining uncertainty, and corrective work are documented. | The relevant failure path is still unexplained. |
| Current authority | Identities, credentials, approvals, policy, and recovery state are current. | Authority depends on restored or stale state. |
| Execution enforcement | The agent cannot bypass the intended gate through another tool, worker, credential, or administrative path. | A known alternate path remains reachable. |
| Behavioral regression | Incident cases and relevant neighboring cases pass under the candidate model, prompt, tools, context, and policy. | The known failure or adjacent failure still reproduces. |
| Evidence integrity | Required target observations and action records are available and interpretable. | Success would rely primarily on the agent’s own report. |
| Intervention readiness | Stop, demotion, human review, and recovery mechanisms work under representative conditions. | Operators cannot reliably interrupt or contain execution. |
| Operational utility | Legitimate actions succeed within the restricted mode. | The control design is safe only because everything fails. |
These are gates rather than weighted scores.
Strong performance in five areas should not compensate for a bypassable execution boundary.
Fix the Mechanism, Not Only the Prompt
If the incident occurred because an approval was missing, strengthening the prompt to “always ask for approval” is not a sufficient repair.
The relevant execution path should require a valid approval artifact regardless of what the model says.
If the incident occurred because the agent trusted a corrupted source, adding “be skeptical” to the prompt is weaker than obtaining decision-critical facts through a protected source path.
Behavior shaping is useful.
Authorization requires enforcement.
Re-run the Incident as a Regression Case
The incident should become part of the qualification suite when it can be represented safely and meaningfully.
That does not mean preserving sensitive production data as a test fixture.
Create the smallest synthetic or sanitized scenario that preserves the failure mechanism.
Then add neighboring variants.
If the incident was:
approved target A agent changed target B
do not create only one test asserting that B is rejected.
Also test:
approved target A, altered target after approval approved target A, stale approval approved target A, changed tool approved target A, broader resource group approved target A, missing source evidence approved target A, delegated worker
Anthropic distinguishes capability evaluation from regression evaluation. That is useful here. After an incident, the known failure should become a regression obligation, while harder adjacent cases continue testing whether the design understands the broader boundary.
A system that memorizes one forbidden example has not necessarily corrected the control problem.
Canary the Autonomy Boundary, Not Just the Software
Canary deployments are commonly used to expose a limited amount of production traffic to a change before widening rollout.
Apply the same idea to authority.
Do not restore an action class across every tenant, resource, user, region, or batch immediately.
Constrain the initial production exposure.
| Canary dimension | Example constraint |
|---|---|
| Resources | One noncritical service group. |
| Tenants | One approved internal tenant. |
| Batch size | One action per request. |
| Rate | Five qualifying actions per hour. |
| Time | Named support window with qualified responders present. |
| Tool set | One narrow action-specific executor. |
| Impact | Reversible internal workflow only. |
| Reviewer | Human confirmation retained for exceptions. |
This is an architectural application of canarying, not a claim that AI autonomy is identical to software release rollout.
The goal is the same in one important respect: expose the changed system to real operating conditions while limiting how much can be affected if the assumption is wrong.
Canarying Needs a Control Group or Baseline
A canary with no comparison can produce false reassurance.
Compare the restricted agent behavior against something meaningful:
- the previous accepted baseline
- explicit policy expectations
- a human-operated path
- a deterministic test set
- similar actions outside the canary
- expected denial rates and permitted-task outcomes
Do not use “nothing bad was reported” as the only success criterion.
The relevant events may be rare, monitoring may be incomplete, or the canary may not yet have exercised the failure condition.
Measure Exposure, Not Quiet Time
A seven-day soak period sounds rigorous.
It may represent almost no evidence if the action was invoked twice.
Measure what the agent actually encountered.
For each requalification stage, track at least:
| Measure | Why it matters |
|---|---|
| Eligible actions observed | Shows the denominator behind the stage. |
| Legitimate actions completed | Prevents denial-everything from appearing successful. |
| Policy denials | Reveals how often the boundary is being challenged. |
| Unsupported permits | Identifies actions allowed without required authority or evidence. |
| Prohibited effects | Measures actual control failure, not only bad recommendations. |
| Unresolved outcomes | Shows where execution or verification remains uncertain. |
| Human overrides | Indicates where automated decisions still require challenge. |
| Stop-path effectiveness | Establishes whether intervention actually blocks dispatch. |
| Evidence completeness | Shows whether the operating claim can be reconstructed. |
Keep model-quality metrics separate from control metrics.
A model may produce a poor recommendation that the execution gate blocks correctly.
That is a model failure and a control success.
Do not average them into one score.
Define Demotion Rules Before Promotion
A progressive autonomy design is incomplete if it explains only how authority expands.
Define what causes it to contract.
Demotion triggers can include:
- a prohibited effect
- approval bypass
- unexplained tool invocation
- missing mandatory evidence
- unavailable execution gate
- stale or unknown policy
- target attribution failure
- abnormal batch or rate behavior
- repeated human override
- retrieval or memory contamination
- unexplained change in decision distribution
- stop mechanism failure
- material model, tool, prompt, or policy change
- an incident in a shared dependency relevant to the action
Not every event requires shutting down every AI function.
Demote the affected action class to the lowest level that remains defensible.
A firewall-change incident might move production network modification to Draft while allowing diagnostic reads to continue.
Demotion Should Be Enforceable
A dashboard status saying restricted is not enough.
The underlying credentials, policy, gateway, tool registry, or execution controller must enforce the new tier.
The agent should not be able to choose to ignore its own demotion.
Human Review Is a Temporary Control Only When the Architecture Supports Leaving It
After an incident, organizations often add manual approval to everything.
That can be appropriate during requalification.
It can also become permanent review theater if the underlying boundaries are never improved.
The question is not whether humans should eventually disappear.
It is whether the remaining action can be bounded strongly enough that per-action human judgment is no longer required.
For some actions, the answer should remain no.
Irreversible, externally binding, security-critical, regulated, or unusually broad operations may belong in a human-owned workflow even after every technical issue has been fixed.
For lower-impact actions, human review can serve as an intermediate qualification stage while the team gathers evidence about proposals, tool use, policy decisions, and outcomes.
DTD’s human-review implementation pattern is useful here because approval should remain bound to the exact action, not become a generic statement that the recovered agent is trusted again.
Use a Requalification Record Instead of a Verbal Go-Live Decision
Represent the requalification state explicitly.
The following YAML is a proposed governance artifact. It does not grant credentials, change a policy engine, or authorize execution by itself.
record_version: "0.1" record_type: agent_autonomy_requalification agent_ref: network-operations-agent incident_ref: incident-2026-041 candidate_baseline_ref: behavior-baseline-019 action_class: operation: service_connectivity_change environment: production requested_autonomy: bounded_execute current_autonomy: assisted_execute qualification: incident_mechanism_addressed: true current_authority_verified: true execution_bypass_tests: passed incident_regression_suite: passed independent_target_verification: passed stop_path_exercised: passed unresolved_related_actions: 0 canary: permitted_resources: 2 maximum_batch_size: 1 maximum_actions_per_hour: 5 required_support_window: staffed promotion: decision: not_established decision_owner_ref: null evidence_bundle_ref: qualification-019 demotion_triggers: - prohibited_effect - missing_required_evidence - approval_bypass - stop_path_failure - unknown_target_outcome
The important value is:
decision: not_established
Passing technical checks does not make a local YAML file a production permission token.
The actual enforcement system should obtain the accepted operating tier from a governed policy source and reject action classes outside it.
Version the Entire Behavioral Baseline
Record the combination that was tested:
- model identifier
- prompt or instruction baseline
- retrieval and context configuration
- tools and schemas
- policy version
- runtime version
- evaluator configuration
- relevant memory state or generation
- execution adapter
- evidence profile
A repaired tool paired with a changed model and changed retrieval configuration is not the same system the regression suite previously assessed.
Requalification belongs to the deployed combination.
Promotion Should Narrowly Expand Authority
Suppose an agent originally had five autonomous write actions.
One action caused the incident.
After repair, do not automatically restore the other four simply because they did not appear in the incident report.
Review whether the failure mechanism could affect them.
There are three reasonable outcomes.
Unrelated action class: Evidence shows the failed dependency is not present. The existing authority may remain valid.
Shared dependency: The action relies on the same model behavior, context source, credential, policy, tool adapter, or evidence path. Requalification should include it.
Unknown relationship: Treat the uncertainty as an assessment gap. Do not resolve it in favor of broader authority merely because the action has not failed yet.
This is the practical use of the Assurance Independence Model during recovery.
Ask which controls and dependencies the action shares with the failed path.
Avoid Common Return-to-Service Traps
“The Root Cause Was the Model”
Sometimes it was.
Sometimes the model exposed a deterministic control weakness that would have been dangerous regardless of model quality.
A model that proposed an unauthorized action should not have been able to convert that proposal into an effect if the action required external enforcement.
Fix both layers when both contributed.
“The New Model Scores Better”
A stronger benchmark result is useful.
It does not establish that identity, authorization, execution, evidence, and recovery boundaries work.
Do not use model quality to compensate for weak architecture.
“No Incidents for Thirty Days”
How many consequential actions occurred?
Which failure conditions were exercised?
Which target classes were reached?
Which cases required human override?
Elapsed time without denominators is weak requalification evidence.
“Humans Approved Everything”
That may show the team is cautious.
It may also show the workflow never demonstrated bounded autonomy.
Measure whether human reviewers detected bad proposals, whether permitted actions completed, and whether the technical controls behaved independently of approval.
“Compensation Worked”
Successful compensation repairs a known effect.
It does not prove that the agent is unlikely to repeat the failure or that another control would catch it.
Recovery evidence and autonomy evidence overlap, but they are not identical.
“We Restored the Old Permissions”
That is precisely what should not happen automatically.
Credentials and policy should reflect the newly accepted operating tier.
Assign Ownership to Promotion and Demotion
No single AI team should both repair the incident and unilaterally restore every previous privilege.
| Decision | Accountable function |
|---|---|
| Candidate model and runtime baseline | AI platform or application owner |
| Action risk and autonomy classification | Architecture, security, governance, and service owner |
| Tool and execution enforcement | Platform or integration owner |
| Identity and credential scope | Identity and security owner |
| Regression and adversarial evaluation | Evaluation engineering with domain experts |
| Target-state verification | Target-system owner |
| Incident closure and remaining uncertainty | Incident authority |
| Human-review readiness | Domain and operations owners |
| Final autonomy promotion | Named service or risk authority |
| Automatic demotion controls | Platform and security engineering |
The exact organizational chart can vary.
The decision rights cannot remain ambiguous.
The team that wants throughput restored fastest should not be the only function deciding whether the evidence is sufficient.
Use a Practical Requalification Runbook
The following sequence is deliberately conservative for consequential production actions.
| Stage | Agent authority | Required exit evidence |
|---|---|---|
| Contained | No consequential execution. | Incident boundary, current identities, and affected actions established. |
| Observe | Read and analyze approved sources. | Context, retrieval, data access, telemetry, and baseline behavior accepted. |
| Draft | Generate real proposals without execution. | Proposal quality, scope discipline, and incident regressions demonstrated. |
| Assisted Execute | Exact human approval before each action. | Execution controls, observations, recovery, and reviewer workflow demonstrated. |
| Bounded Execute | Limited autonomous actions inside hard constraints. | Representative production evidence, canary metrics, stop behavior, and low unresolved outcome rate. |
| Expanded Autonomy | Wider action population only where justified. | Sustained evidence across representative cases and accepted residual risk. |
The stages do not need equal duration.
A low-impact action might move quickly.
A security-critical or partially reversible action may remain at Assisted Execute indefinitely.
Promotion Is an Explicit Decision
Do not make promotion depend only on a timer.
Use an evidence bundle and named decision owner.
The owner should be able to answer:
What changed?
Which failure mechanism was addressed?
Which cases were tested?
Which production actions were observed?
What remains unknown?
What new blast radius does this promotion create?
What event will automatically reverse the promotion?
If those answers cannot be stated clearly, the next autonomy tier is premature.
When Previous Autonomy Should Not Return
Sometimes the incident reveals that the earlier autonomy level was simply too ambitious.
Do not treat that conclusion as a failed recovery.
Keep the lower operating mode when:
- the action remains difficult to reverse
- target-side evidence cannot support reliable verification
- independent enforcement cannot be implemented
- the required approval cannot be bound to the exact action
- the human stop path cannot act before unacceptable harm
- the relevant data or context cannot be independently established
- the tool remains too broad
- unresolved shared dependencies remain
- legitimate operation requires privileges larger than the accepted blast radius
- compensation leaves material residual harm
- the action is externally binding or regulated in a way that requires accountable human ownership
An agent can remain highly valuable in Draft or Assisted Execute.
The goal is not maximum autonomy.
The goal is the highest autonomy the evidence can defend.
Conclusion
The final stage of AI incident recovery is not turning the agent back on.
It is deciding what the recovered system is again permitted to do.
That decision should be action-specific, evidence-backed, progressively exposed, and easy to reverse when new information appears. A corrected model, repaired tool, successful compensation, or restored service is part of the evidence. None of those alone restore the previous assurance case.
Use real regression cases. Exercise the enforcement boundary. Canary the authority. Measure actual exposure rather than elapsed time. Define demotion rules before promotion. Preserve human review where the residual blast radius still requires it.
Most importantly, do not make yesterday’s autonomy level the default recovery target.
The control plane must not grade itself. Authority should return only as fast as independent evidence can justify it.
Choose one consequential action your agent performed before an incident. If you had to earn that autonomy again from zero today, what evidence would you require before removing the human approval?
External References
- NIST: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
- NIST SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management: A CSF 2.0 Community Profile
- OWASP: AI Agent Security Cheat Sheet
- Google SRE Workbook: Canarying Releases
- Anthropic: Demystifying evals for AI agents
Choose reversal, compensation, forward recovery, or containment according to the observed effect. Authorize corrective actions, preserve concurrent changes, and track consequences that...
1 thought on “Restoring AI Agent Autonomy After an Incident”