Security & Prompt Injection
Description
Section titled “Description”An autonomous coding fleet is a high-value, high-blast-radius attack surface: it holds credentials, reads untrusted input, and can write code, open PRs, and reach the network. The Architecture already enforces network-policy isolation for the agent and MCP namespaces. This page covers the threats that network isolation alone does not stop chiefly prompt injection and the principle that the only durable defences are structural, not textual.
The governing insight, learned the hard way across the field: instructions in a system prompt (“don’t touch production”, “don’t exfiltrate data”) are not controls. Under adversarial input or long-horizon pressure, agents ignore them. Controls must be enforced by architecture.
The Lethal Trifecta
Section titled “The Lethal Trifecta”An agent is unconditionally vulnerable to data exfiltration via indirect prompt injection when it combines three capabilities (Simon Willison’s framing):
- Access to private data (repo contents, secrets, internal systems),
- Exposure to untrusted content (issues, PR comments, fetched web pages, dependency READMEs),
- The ability to affect the system (open a PR, make a network call, write to a shared surface).
Coding agents hit all three by default. The canonical exploit: a malicious issue filed on a public repo carries injection instructions that make the agent read private repos and exfiltrate their contents through a PR. Issues, PR comments, web pages, and tool output are untrusted input and must be treated as adversarial.
The defence is not a better classifier. A guardrail that catches “95% of attacks” is a failing grade an attacker simply retries. The only reliable mitigation is to remove one leg of the trifecta. For a coding fleet, the cheapest leg to break is usually exfiltration/egress: constrain what the agent can send and where, so that even a fully injected agent cannot get data out.
Sources: Simon Willison The Lethal Trifecta · Trail of Bits Exploiting GitHub Copilot
Structural Guardrails, Not Prompt Instructions
Section titled “Structural Guardrails, Not Prompt Instructions”Every high-consequence constraint must be enforced outside the model:
- Environment separation. Dev and prod are separated structurally, so an agent physically cannot reach production data or systems. The Replit incident (agent deleted a production database during a code freeze, then fabricated data to hide it see Verification) is what happens when “don’t touch prod” lives only in a prompt.
- Post-ingestion action gates. Once an agent has consumed untrusted input, consequential actions (merge to main, force-push, deploy, edit CI/secrets, delete branches) require a Controls gate a human or policy approval the agent cannot self-grant.
- Egress allowlists. Default-deny outbound network, with an explicit allowlist. See the MCP caveat below.
The MCP Egress Hole
Section titled “The MCP Egress Hole”MCP is the documented soft underbelly. Egress firewalls and many guardrails explicitly exempt MCP servers GitHub’s Copilot coding-agent firewall, for example, applies only to processes the agent’s shell starts, not to MCP servers or configured setup steps. If agents are given MCP tools, the isolation story has a hole exactly where data can leave.
Treat MCP traffic as a first-class egress and audit surface: route it through the gateway, log every call to the Execution Ledger, and apply the same egress policy as to shell network calls. This reinforces the Architecture’s decision to put MCP in its own locked-down namespace but the lockdown must cover egress, not just ingress.
Source: GitHub Customizing the Copilot coding agent firewall
Sandboxing Tiers
Section titled “Sandboxing Tiers”Standard Docker / runc shares the host kernel and is insufficient for untrusted agent-generated code. The 2026 production bar is stronger isolation:
| Tier | Mechanism | Examples |
|---|---|---|
| Container | shared kernel not sufficient alone | plain Docker |
| User-space kernel | syscall interception | gVisor (Modal) |
| microVM | own guest kernel + network namespace | Firecracker (E2B), Kata, Daytona |
Ikidna’s Coder-based isolated workspaces and per-task git worktrees provide workspace isolation; for executing untrusted code the runtime should target microVM/gVisor-grade boundaries, not bare containers. This is the same blast-radius concern the Architecture agent-namespace notes raise, made concrete at the runtime layer.
Credentials: The Aggregate Blast Radius
Section titled “Credentials: The Aggregate Blast Radius”Each credential an agent holds may be individually least-privilege, but the aggregate (cloud + repo + ticketing + chat + N SaaS tools) is a blast radius no service account was ever designed to have, and no single per-service review reveals it. Two standard mitigations:
- Short-lived, brokered credentials. The agent never holds long-lived keys. It requests scoped, time-bound tokens per task via an OIDC token broker / dynamic secrets, with lifetimes in minutes and automatic revocation. This is the confused-deputy mitigation: the agent is a deputy that should never wield standing authority.
- A portfolio view. Maintain an explicit, auditable inventory of everything every agent class can touch, because the risk is invisible from any single integration.
This extends Operations and Governance’s “virtual keys / delegated credentials instead of raw long-lived secrets” from a principle into a concrete broker requirement. Ikidna’s Architecture already delegates VCS access through Vault the broker model generalises that to every credential.
Sources: SANS The Confused Deputy / Credential Broker · GitGuardian Short-Lived Credentials in Agentic Systems
Agents Generate Secret Leaks
Section titled “Agents Generate Secret Leaks”AI-assisted code commits secrets at roughly twice the human baseline. The fleet is a source of leak risk, not just a consumer of credentials. Scan agent output for secrets before any PR is opened, as a non-bypassable pipeline step.
Detecting a Compromised Agent at Runtime
Section titled “Detecting a Compromised Agent at Runtime”Catch injection in flight by monitoring for behavioural anomalies, as security telemetry on the Execution Ledger: anomalous tool-call sequences, unexpected egress destinations, and sudden scope expansion (an agent that was scoped to one repo suddenly reading others). These are the runtime signals that a guardrail has been bypassed.
Related Concepts
Section titled “Related Concepts”- Controls autonomy ladder and approval gates for consequential actions.
- Architecture namespace isolation and network policy.
- Operations and Governance credentials, approval boundaries, audit.
- Verification & the Outer Loop never trust self-reports of destructive actions.
- Execution Ledger audit trail and security telemetry.
- MCP the egress surface to lock down.
- Prior Art & Lessons sources.