The audit agent had been blind for ten consecutive days before I noticed. Its scheduled job fires every morning, cross-checks the operation’s shared board against repositories and logs, and opens a pull request with its findings. The pull requests kept arriving, so the system looked alive. Inside each one: nothing, because the agent could no longer reach the data it was supposed to audit. Nineteen of its reports piled up unreviewed.
That is the state of the art in AI agent governance that nobody writes about. The literature worries about agents doing something dangerous. My agents never went rogue. They went quiet. The same operations pass that caught the blind auditor found a second problem: the spec-drafting daemon had been switched off by a configuration batch 47 days earlier, and no alarm existed to say so. The machinery was fine both times. What failed was feeding and watching. This piece is the architecture, the governance model, and the six failures that taught me more than the design did.
The Operation: Three Agents and One Human
The operation runs the back office of a one-person consulting practice: research, pipeline hygiene, content drafting, internal tooling, and the janitorial work that eats evenings. Three agents with deliberately unequal privileges do that work. I remain the only judgment node.
The strategist (internal name: Alfred) is the orchestrator. It holds full business context, plans work, drafts everything a human will read, and routes tasks to the other two. It is the only agent with access to sensitive data, and everything it produces for the outside world stops at a draft.
The executor (Lucius) is a spec-bound coding agent, and it is narrow on purpose. No tools beyond the repository it works in, no memory between tasks, no access to business data. It consumes written specs from a file queue, works in an isolated git worktree, pushes a branch, and never commits to the main branch. If a spec is ambiguous, it refuses and parks the task in a blocked queue with questions.
The auditor (Oracle) is a scheduled cloud job that cross-checks the shared work board against what actually happened in the repositories. It cannot mutate anything. Its entire output surface is a pull request describing drift and proposing corrections.
They communicate through boring, inspectable substrates, and deliberately not through a message bus. Work moves through a file queue with inbox, in-progress, done, and blocked directories, plus an append-only log. Findings and proposals move as pull requests. Shared state lives on a project board that tracks roughly eighty work items across a dozen internal product lines. Every hop between agents is a file, a branch, or a card that a human can read three weeks later. When something breaks, the forensic trail is already written.
The three agents also run on models from three different providers. That started as a cost decision and became a governance feature, for reasons that show up below.
The Governance Model That Held
The design rule underneath everything: the more autonomous an agent is, the less it should be able to touch. Privilege scales inversely with autonomy. Four mechanisms carry most of the weight.
The exfiltration triangle. An agent that combines attacker-controlled input, autonomous egress, and access to sensitive data is one prompt injection away from leaking everything, regardless of which model runs it. Any two of the three legs are acceptable. All three are not. So every agent in the operation has one leg structurally removed. The executor reads untrusted repository content and can push branches, so it holds no sensitive data. The triage prototype that read my full inbox had its egress cut instead: a hostname-allowlist proxy, no send credential, no fetch tool. The point is that safety comes from architecture, not from trusting a model to resist a well-crafted injection.
Tiered autonomy with one master switch. Every automated leg is classified SENSE (reads only), DRAFT (produces something a human approves), or EXECUTE (acts without a human in the loop). SENSE and DRAFT run all day and can never touch the outside world. EXECUTE is gated by a single arm file in the repository: presence means armed, deleting it is an instant stop, and its state is visible in version control like everything else. Kill switches are presence files rather than config entries because presence is diffable, greppable, and impossible to misread during an incident.
Drafts-only sends. No model ever sends an outbound message. Anything client-facing terminates in a queue, and a model-free script fires only on a human-set approved flag. The send step is deliberately too dumb to be talked into anything.
Pull-request merge as the approval contract. Every proposal, whether a code change from the executor or a board correction from the auditor, arrives as a pull request. Merging is the approval signal. Closing with a comment is rejection. This one convention does an enormous amount of work: the audit trail, the diff review, the ability to hold a specific change while approving the rest, all inherited from mature tooling instead of built.
Around those four sit the supporting rules. A routing firewall decides what the executor may receive: if acceptance criteria cannot be written as tests pass, build green, a grep returns the expected count, and lint is clean, the task never reaches it. Voice-carrying copy can never satisfy that test, which is exactly the intent. Every substantial diff gets reviewed by a model from a different provider than the one that wrote it, because same-model self-review approves its own blind spots. An escalation ladder runs auditor to orchestrator to human with no skipping. A short list of forbidden paths, instruction files, voice and identity surfaces, agent memory, is writable by no agent at all. And every automated action sits inside a seven-day human veto window, enforced by a weekly backward scan of the logs.
What Broke
The design above reads clean. The year did not. Six incidents, each with the lesson it forced.
Week one: a 67 percent refusal rate. The first three specs through the executor queue produced one completed task and two refusals on acceptance criteria. The causes were unglamorous: a sandbox configuration error and a mismatch between the spec and the executor’s forbidden-path list. The following week the refusal rate fell to zero and average time from spec to pushed branch was about 25 minutes. Nothing about the model changed. The specs changed. Spec quality is the failure surface, not the executor.
Stacked reports that conflict with each other. The auditor’s daily pull requests all appended findings to the same log file. After the first merge, every subsequent open report conflicted. The fix was procedural rather than clever: clean reports are closed with a comment instead of merged, since a no-findings report proposes no changes anyway. The general form: agents that share a write target will collide on it. Design agent outputs to be conflict-free, or accept close-not-merge as the norm.
The near-miss the gate caught. The auditor once proposed advancing a work item to ready-to-ship on a repository that deploys automatically on merge. The forbidden-action list forced an escalation instead of an auto-advance, and a human looked at it. That particular card turned out to be legitimate. On a different day, the same auto-advance is an unreviewed production deploy. The lesson is about where the gate sits: it held because deploy-on-merge repositories were on the forbidden list from the start.
The trust-model design bug. The inbox triage prototype was designed to protect privacy by stripping personal data before the agent saw anything, reducing each message to categories and flags. A live test showed the projection starved the task: you cannot draft a useful reply from enums and booleans. The fix flipped the trade. The agent now reads full messages, and safety moved to a structural egress cut, the allowlist proxy and missing send credential described above. For triage and assistant workloads, cut the egress leg, not the data leg.
A prompt-injection path to shell access. A security review of a headless composer that summarized untrusted inbound text found it had been provisioned with shell tools. Malicious input to that agent was a plausible path to code execution on the host. It was rebuilt as a pure text composer, with a wrapper script doing all input and output. The rule that survived: a headless agent over untrusted input gets no shell, no permission overrides, no exceptions.
The automation that never got armed. The highest-blast-radius tier, board-wide auto-actions across every status of every work item, has roughly ten times the surface of the auditor’s narrow triage. It shipped behind its own kill switch, ran in dry-run validation, and weeks later it is still off. Meanwhile the silent failures described at the top happened anyway, in components that were armed. Restraint turned out to be a governance feature: blast radius, not capability, decides the arming order, and some switches deserve to stay off longer than enthusiasm wants.
The thread through all six, and through the two silent outages in the opening: none of these were model failures. They were specification failures, coordination failures, provisioning failures, and monitoring failures. The failure mode was never the machinery. It was feeding and watching.
One Boundary, for the Record
I lead enterprise CRM transformation programs for a living. That is the day job, and it is not agent-run. The operation described here is my own back office, one operator and a work board, not a client deliverable. The governance lessons transfer because they are architectural. The scale does not, and I will not pretend otherwise. If you are evaluating agent platforms inside the Salesforce stack, the same governance-before-scale sequence applies one layer up, in how Agentforce Builder changes the authoring risk profile.
The Minimum Governance Kit
What I would put in place before letting any agent act, distilled from the incidents above. Each item works on its own.
- One master arm file. A single presence flag gates every leg that can act without a human. Deleting one file stops the fleet.
- Drafts-only sends. No model holds a send credential. A model-free step sends on a human-set flag.
- A mechanical approval contract. Proposals arrive as pull requests; merge means approved; close means rejected. No verbal approvals.
- A routing firewall for executors. If acceptance cannot be encoded as tests, builds, and greps, the task stays with a human-supervised agent.
- Cross-model review. A different provider’s model reviews every substantial diff before merge.
- An escalation ladder with no skipping. Auditor to orchestrator to human, one step at a time, so context accumulates instead of evaporating.
- A kill switch per leg. Presence files, one per automated component, next to the master arm.
- A forbidden-path list. Instruction surfaces, identity and voice files, and agent memory are writable by no agent.
- Silence detection. Every scheduled component gets a heartbeat and an alarm on absence. Silence is the default failure mode, and it is the one no model will report.
- A human veto window. Every automated action is reversible for seven days, and a weekly scan actually looks.
Key Takeaways
- The dominant failure mode of an autonomous agent operation is silence, not rogue action; an audit agent was blind for 10 consecutive days and a drafting daemon dark for 47 before anything noticed, so heartbeats and silence detection belong in the first release, not the backlog.
- Privilege scales inversely with autonomy: the agent that acts the most freely should touch the least, and every agent should have one leg of the exfiltration triangle removed structurally rather than by policy.
- Spec quality, not model quality, is the executor’s failure surface; a 67 percent week-one refusal rate fell to zero without changing anything about the model.
- A pull-request-based approval contract gives audit trails, diff review, and human gates for free; the merge button is the cheapest governance infrastructure that exists.
- Arm automations by blast radius, not by readiness; the widest-surface tier in this operation is still off, and that restraint has cost nothing while the armed components produced every incident worth writing about.
