TL;DR
AI decision controls separate what a model proposes from what an application may execute. A useful recommendation still needs appropriate evidence, current authorization, bounded resource use, and outcome verification. Enforce those requirements outside the model, bind approvals to specific operations, and preserve uncertain execution outcomes instead of treating every timeout as a failed action. Start with a narrow diagnostic workflow and test the entire path, including denied requests, policy changes, retries, and recovery. Better model behavior does not remove these responsibilities.
Introduction
The checkout-service assistant now handles changing evidence more reliably. Its memory distinguishes yesterday’s incident from current observations. Engineers find its recommendations useful and ask for the next improvement: let it collect the diagnostic evidence itself.
The request sounds modest. Instead of recommending a check of the connection between checkout_api and catalog_api, the assistant would select and invoke an approved diagnostic tool.
But which endpoint will it contact? Under whose identity? What information will it return? How often can it run? What happens when the request times out and the assistant decides to try again?
Part 1, AI Generalization: Did Your Model Learn the Right Pattern?, examined learned behavior. Part 2, AI Memory Architecture: Context, RAG, and Persistent State, examined the information supplied to the model. This final article addresses the boundary between a recommendation and an operational commitment.
We will continue the hypothetical incident scenario. The proposed release permits bounded diagnostic calls, not production configuration changes. Model parameters remain fixed during normal inference. The architecture, contract, and tests below are proposed engineering patterns, not a tested implementation or a compliance certification.
A model can recommend the right action and still lack permission to perform it.
Separate the Questions Hidden Inside “The AI Was Right”
Connectionism explains how a neural model can develop useful relationships without an engineer specifying every task rule. It does not establish that a particular response is supported by current observations or authorized for execution.
For this assistant, I would separate five questions rather than reduce them to one confidence score.
| Question | Appropriate check | What passing does not establish |
|---|---|---|
| Is the proposal well formed? | Validate the operation, parameters, and resolved target | That the operation is useful or permitted |
| Is there an adequate reason to act? | Review relevant evidence and the diagnostic objective | A confirmed root cause |
| Is this action authorized now? | Check caller, workload identity, target scope, and policy | That current conditions make execution appropriate |
| Can it run within operating limits? | Check resource budgets, concurrency, and applicable preconditions | That execution will succeed |
| Did it achieve its objective? | Inspect the result and obtain appropriate follow-up evidence | That an API success response proves service recovery |
The evidence requirement should match the operation. Collecting a narrowly scoped observation does not require a confirmed diagnosis; gathering that evidence may be the point. A disruptive remediation needs a different justification.
Likewise, checking that a cited record exists does not prove that the model interpreted it correctly. The application can validate exact constraints while domain review addresses judgments that cannot be reduced to a simple field check.
The operating objective is not to eliminate uncertainty before every action. It is to choose actions whose scope and consequences are appropriate to the uncertainty that remains.
Put the Execution Boundary Outside the Model
OWASP’s LLM06:2025 Excessive Agency identifies excessive functionality, permissions, and autonomy as contributors to damaging actions. Its guidance includes narrow tools, least-privilege access, appropriate human approval, and authorization enforced by downstream systems rather than by the model.
For the checkout assistant, that means the model emits a proposal. The application decides whether the proposal can reach a tool. The downstream service enforces the restrictions it is responsible for.
The diagram shows the critical separation: the model does not supply its own verified identity, issue its own approval, or bypass the executor.
Treat this as an enforcement boundary, not a sequence of advisory prompts. A policy instruction in the model context is not a substitute for a denied API request.
In this design, the model-facing component has neither credentials nor a permitted network path for direct production access. Tool adapters accept only the defined operations. Do not leave a general shell, alternate integration, or broader credential available as a convenient fallback.
Returned diagnostic material also remains data. A string in a tool response must not gain authority to change the policy controlling the next call.
Preserve Caller Scope Through the Whole Workflow
An engineer’s identity and the executor’s workload identity have different purposes. The first establishes who requested the work and what they may delegate. The second identifies the application component performing it.
OWASP’s Authorization Cheat Sheet recommends denying access by default and validating permissions on every request. Apply that principle to the complete tool path, not only the initial chat session.
For this proposed workflow, the application derives caller scope from authenticated context and resolves targets through a maintained service inventory. It then checks whether the requested diagnostic is permitted for that caller, workload, and target combination.
A dependency relationship is not an access grant. Permission to investigate checkout_api does not automatically authorize collecting restricted information from every service it calls.
The same rule applies when work moves to another agent or background worker. Preserve the initiating request’s constraints and evaluate the receiving component’s authority. Do not let delegation become a route to a more privileged shared account.
A model-generated approved: true, tenant name, or incident identifier is input to validation, not evidence of authority.
Define a Small, Inspectable Action Contract
For the first release, I would allow one purpose-built operation that collects approved TLS connection metadata. The tool would not expose arbitrary commands, packet payload collection, or configuration changes.
The following YAML is an illustrative architecture contract. It is not deployable configuration for an existing policy engine, and placing it in a prompt would not enforce it.
contract_id: checkout_incident_diagnostics revision: 1 default_decision: deny allowed_operation: name: collect_tls_metadata version: 1 target_set_ref: checkout_approved_dependency_endpoints collection_context_ref: checkout_client_probe_profile output_profile_ref: tls_metadata_no_payload max_collection_seconds: 15 max_requests_per_incident: 3 requirements: caller_scope: current_authorization workload_scope: least_privilege policy_check: before_each_dispatch shared_budget_ref: production_diagnostics operation_record: durable_before_dispatch on_unavailable_required_control: deny_and_escalate production_configuration_changes: denied
Replace the operation, target set, collection context, output profile, and policy references with approved application definitions. The fifteen-second duration and three-request limit are illustrative values, not production defaults.
The collection context should specify where the observation is taken and which connection configuration it represents. Otherwise, an apparently successful probe might not answer the question about the checkout workload’s actual path. The output profile should explicitly define permitted fields and record that context with the result.
The adapter must implement collection limits; a client-side timeout alone does not establish that remote work stopped. It must also reject unsupported parameters rather than quietly passing them to a more flexible backend.
Successful implementation would allow the approved diagnostic within its scope while blocking unsupported operations, unauthorized endpoints, unavailable required controls, and exhausted budgets. Those outcomes require integration tests, not merely successful YAML parsing.
Enforce Shared Limits, Not Just Per-Agent Counters
In this design, the application binds requests to a trusted incident record. A model cannot obtain a new allowance by inventing an incident identifier or delegating to another worker.
Reserve shared capacity before dispatch using concurrency-safe accounting. A read-then-increment counter can admit competing requests against the same remaining allowance. Define limits across the protected service or tenant as well as the individual workflow.
Count attempts separately from completed jobs. A stream of unsuccessful requests can still consume resources. When a limit is reached, the fallback is escalation or a later authorized attempt, not permission for the model to raise its own budget.
Bind Human Approval to the Actual Operation
Human review needs a defined object. “Proceed with the investigation” is too broad to authorize every subsequent tool choice.
OWASP’s Transaction Authorization Cheat Sheet recommends confirming significant transaction details, protecting them against modification, and placing a final authorization check at execution. Its transaction-security guidance provides a useful foundation for an AI approval interface.
For this assistant, I would present the resolved target, operation version, parameters, output destination, expected impact, and validity period. The review interface should obtain those details from the application-held operation record, not only from the model’s narrative summary.
Bind the approval to that record. Material changes require renewed review, and the executor must verify that the approver has the required authority. A hash alone is not proof of approval; its trusted association with the approval decision matters.
Approval also remains subject to policy. In the initial release, a human clicking “approve” must not enable a production mutation that the contract prohibits. Broader operations require a separate, reviewed release rather than an exception invented inside the conversation.
Revalidate Authority When Work Reaches Execution
A queued diagnostic can outlive the conditions under which it was requested. The caller’s access may be revoked, the endpoint may move to a different ownership boundary, or the incident may be closed before dispatch.
For this architecture, record the policy revision used for each decision and re-evaluate material conditions when work reaches the executor. A saved decision records what was allowed earlier; it is not an indefinite grant. Define the revocation propagation requirement separately from the ability to cancel already accepted work.
This does not require every implementation to call a remote policy service for every check. An approved local policy snapshot can be part of the design. Its freshness, revocation behavior, and failure policy still need explicit limits.
If a required control is unavailable, this diagnostic pilot denies the new dispatch and escalates. That can delay investigation, so maintain an independently authorized human runbook. Do not resolve the availability tradeoff by silently giving the assistant broader permissions.
A Pre-Execution Check Does Not Remove Every Race
A later write-enabled design would also need resource-side concurrency safeguards. Kubernetes provides a concrete example: for an HTTP PUT updating an object, the API server uses resourceVersion to detect an outdated client and returns 409 Conflict when the supplied version is stale. Appropriate conditional patch mechanisms can also detect conflicting state.
That protects a particular update against stale object state. It does not make a policy decision, an approval, and a multi-service workflow one atomic transaction.
For an approval-sensitive operation, a conflict should trigger reassessment of the current state. Do not simply refresh the version and replay an earlier decision against changed conditions.
Production writes remain outside this pilot. The example illustrates why expanding authority requires more than adding another tool name to the allowlist.
Preserve Unknown Outcomes Instead of Blindly Retrying
A timeout does not prove that the downstream service did nothing. The request may have been accepted even though its response never reached the caller.
The Amazon Builders’ Library article Making retries safe with idempotent APIs explains how service-supported request identifiers can distinguish repeated attempts from new intent. Its design also rejects reuse of an identifier with conflicting parameters.
For this assistant, keep the same operation identity when reconciling an uncertain attempt. A fresh identifier could make the next request appear to be new work. Confirm the executor’s actual idempotency scope and retention window; an identifier written only into the assistant’s log does not provide that guarantee.
Where the service cannot reconcile or safely deduplicate an uncertain operation, escalate rather than automatically repeat it. Do not describe the whole workflow as exactly-once execution merely because one component supports idempotency.
The proposed state model keeps uncertainty visible instead of converting it into failure or success.
Store these transitions in application workflow state, not in a conversation summary. A journal entry proves intent, not remote execution. Recovery after a worker restart must reconcile unresolved operations rather than infer success or assume nothing happened.
For the diagnostic pilot, completion means the intended metadata was collected from the approved context and returned within scope. It does not mean the root cause was proven or the checkout service recovered.
Keep Decision Evidence Separate from Execution Evidence
An authorization record establishes what the policy evaluator decided. An execution record establishes what the adapter attempted and what the downstream system reported. Follow-up observations establish whether the operational objective was achieved.
Correlate those records, but do not treat them as interchangeable.
Open Policy Agent’s Decision Logs documentation provides a concrete example of recording policy inputs, decisions, and bundle revisions, with mechanisms to mask sensitive data. It also documents conditions under which events can be dropped. Such telemetry should not be assumed to be a transactional record of every operational action.
For this design, use a durable operation journal and correlate it with policy decisions, model and prompt versions, approved targets, evidence identifiers, and observed results. Define how the executor behaves when it cannot preserve required records.
Protect that evidence without retaining every prompt or raw diagnostic indiscriminately. Restrict access, apply retention rules, and record enough structured information to reconstruct the decision and execution path.
An approval record is not proof of correctness. A successful tool response is not proof of recovery. The operating record should preserve those limits rather than hide them behind one “completed” flag.
Test the Boundary Even When the Model Behaves Badly
Part 1 tested whether recommendations changed appropriately with evidence. Part 2 tested whether memory could be corrected, expired, and revoked. Add enforcement and recovery tests before enabling model-selected tool calls.
Use both end-to-end scenarios and direct requests to the application and executor interfaces. A model refusing to propose an unauthorized action does not demonstrate that the backend would block that action.
| Test | Expected behavior |
|---|---|
| Valid diagnostic within scope | The approved operation completes and produces correctly scoped evidence |
| Unauthorized target or alternative tool route | Enforcement blocks the request independently of the model’s response |
| Caller access revoked while work is queued | The executor denies the new dispatch under current access conditions |
| Material parameters changed after approval | The altered operation cannot reuse the earlier approval |
| Parallel requests exhaust shared capacity | Aggregate admission stays within the defined budget |
| Timeout after downstream acceptance | The journal preserves uncertainty and the workflow reconciles without creating unintended duplicate work |
| Policy or required journal unavailable | The pilot follows its defined denial and escalation path |
Run the tests in an isolated environment with controlled identities, clocks, policy revisions, and backend fault injection. Inspect actual downstream calls and state transitions, not just the assistant’s final wording.
NIST’s Generative Artificial Intelligence Profile recommends verifying output sources, documenting generalization limits, and reviewing security and safety controls, particularly under new operating circumstances. The matrix above applies those concerns to this proposed workflow; it is not a NIST certification checklist.
Define consequential failures before evaluating a release. Unauthorized execution should not disappear inside a strong average task-completion score. Also measure unnecessary denials, diagnostic usefulness, control latency, and escalation quality so that containment does not become a substitute for service value.
Treat Model Changes as Operating Changes
For this application, a model replacement is a reason to re-evaluate the action boundary even when permissions remain unchanged. A different model may choose different tools, construct different arguments, or pursue a workflow more persistently. That is a design risk to test, not an assumption that every upgrade increases danger.
Use the same release discipline for material changes to prompts, retrieval, memory, tool contracts, and policy. Where possible, isolate the changed component first, then test the integrated service.
The service owner defines acceptable actions and recovery conditions. Security owns authorization requirements. The application and platform teams implement enforcement and durable execution. The model team evaluates behavior. Those responsibilities need one release decision and a shared incident process.
Start with proposal-only evaluation, progress to the bounded diagnostic pilot, and expand only when the evidence and operating capability support a specific additional action. High-impact or poorly recoverable operations may appropriately remain human-executed.
Keep containment distinct from rollback. Disabling new dispatches stops additional work; restoring a previous model changes future proposals. Neither automatically cancels accepted jobs or reverses effects already produced. Reconcile in-flight work and recover external state through the responsible operational system.
Conclusion
Connectionism explains the learned relationships that make neural AI useful. Enterprise execution requires a surrounding system that determines whether a proposed action is supported, authorized, bounded, and actually completed.
Across this series, the responsibilities remain distinct. Evaluate learned behavior under meaningful changes. Govern the information supplied through memory and retrieval. Enforce current authority at the action boundary and verify outcomes afterward.
For the checkout assistant, the next practical step is one purpose-built diagnostic operation with an explicit contract, independent enforcement, durable execution state, and reviewed failure tests. That is a stronger foundation than broad tool access combined with increasingly elaborate instructions.
A better model can improve the recommendation. The production system must remain responsible for deciding what may happen and establishing what did happen.
External References
- OWASP Gen AI Security Project: LLM06:2025 Excessive Agency
Canonical URL: https://genai.owasp.org/llmrisk/llm062025-excessive-agency/ - OWASP Cheat Sheet Series: Authorization Cheat Sheet
Canonical URL: https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html - OWASP Cheat Sheet Series: Transaction Authorization Cheat Sheet
Canonical URL: https://cheatsheetseries.owasp.org/cheatsheets/Transaction_Authorization_Cheat_Sheet.html - Amazon Builders’ Library: Making retries safe with idempotent APIs
Canonical URL: https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/ - Kubernetes: Kubernetes API Concepts
Canonical URL: https://kubernetes.io/docs/reference/using-api/api-concepts/ - Open Policy Agent: Decision Logs
Canonical URL: https://www.openpolicyagent.org/docs/management-decision-logs - NIST: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
Canonical URL: https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
TL;DR The double-slit experiment and AI share a useful lesson about interpreting results: the conditions that produce an observation matter. That does...