Self-Learning Agents

The durable advantage in an agent product is not access to the same model API as everyone else. It is the ability to learn from production use. Every agent run and every human correction can become evidence that improves the next run. Without a capture loop, users repeatedly teach the product the same lesson and the lesson disappears when the interaction ends.

At product scale, those interactions form a compounding asset. The condition is simple but demanding: capture both sides of the work, join them into one learning record, and test every proposed lesson before promotion.

Two signals, one outcome

Most learning systems observe only half of the interaction.

Agent traces show what the system tried

An agent trace records the request, context, model and prompt versions, tool calls, tool results, intermediate steps, outcome, latency, and failure state. It can answer:

• What did the agent attempt?
• Which tool path did it choose?
• What evidence did it receive?
• Where did execution fail or stop?
• Which prompt, skill, or harness configuration was active?

Point an analysis agent at a set of traces and it can cluster failure patterns, propose better instructions, identify unreliable tools, and turn production misses into regression cases. The Execution Ledger is the system of record for this side of the loop.

But a trace ends at the agent boundary. It cannot see the user quietly correcting a field, undoing a generated edit, choosing a different workflow, or completing the task manually after dismissing the agent.

Ambient user activity shows what worked

The second signal is the activity around the agent: clicks, edits, approvals, rejections, overrides, follow-up actions, and the final application state. These events reveal what the user accepted and what they repaired.

A correction is especially valuable because it pairs a failed proposal with a better answer. Brex describes this pattern in its AI-native onboarding system: analysts handle uncertain cases, and their decisions return to the system as supervised signals. Each intervention solves the current case and creates a labelled example for future calibration. Brex’s account of its onboarding system also shows the prerequisite: domain experts first made their implicit judgment explicit and built a reviewed benchmark from real cases.

Ambient activity is not ground truth by default. A click may be accidental, an edit may reflect personal preference, and silence does not mean approval. Treat high-confidence actions such as an explicit correction or verified outcome differently from weak behavioural proxies such as dwell time.

The interface sees both

The application interface is where the person and the agent work against the same state. It can observe the agent’s proposal and the user’s response to that proposal, then bind them with a shared run, task, actor, and artifact identity.

The system needs a correlation boundary where agent execution and user activity can be joined. That boundary may be a product-native event bus, an observability pipeline, a durable workflow engine, a shared telemetry contract, or an interaction protocol. No single standard owns this boundary.

AG-UI is one possible implementation at the application-to-agent edge. It is an open, lightweight, event-based protocol for bidirectional communication between user-facing applications and agent backends. Its typed lifecycle, message, tool-call, state, raw, and custom events can carry part of the joined interaction stream. It is transport-agnostic and can sit above SSE, WebSockets, webhooks, or another delivery mechanism.

AG-UI does not automatically turn every browser action into training data, nor must a product adopt it to build this loop. Whatever boundary is chosen, the product must deliberately instrument meaningful user events, define their semantics, collect consent, remove sensitive data, and correlate them with the relevant agent run. Version those event contracts like any other API.

flowchart LR
    A["Agent trace<br/>proposal, tools, result"] --> J["Joined interaction record"]
    U["User activity<br/>correction, approval, outcome"] --> J
    J --> C["Classify the lesson"]
    C --> E["Build or update an eval case"]
    E --> G{"Beats the baseline?"}
    G -->|Yes| P["Promote to context, harness, or weights"]
    G -->|No| R["Retain as evidence"]
    P --> A

The joined record should preserve provenance rather than flattening everything into a transcript. At minimum, retain the run and actor identifiers, event timestamps, before and after state, agent proposal, human correction, verified outcome, active configuration versions, consent class, and redaction status.

One correction can create three memories

A useful interaction can teach the system at three different levels:

1. Semantic memory, the fact. A durable piece of domain knowledge, such as a customer’s approved refund limit.
2. Episodic memory, the case. What happened in this specific situation, including the agent’s attempt, the correction, and the outcome.
3. Procedural memory, the rule. A reusable instruction or policy, such as approve below the limit and flag anything above it.

Do not promote all three automatically. The case is evidence and can usually be stored immediately. A fact needs provenance, scope, and an expiry policy. A rule needs repeated support or an explicit expert decision, followed by evaluation. This prevents one unusual correction from becoming a global instruction.

Where the lesson lives

A captured lesson has three possible destinations. Each buys a different kind of improvement and carries a different cost.

Layer	Best for	Advantage	Cost and risk
In-context	Fresh facts, user preferences, retrieved cases, temporary guidance	Fastest to update and easy to reverse	Consumes context, can distract the model, and must be retrieved correctly on every run
Harness	Tool permissions, workflow steps, routing, prompts, checks, escalation rules	Deterministic, inspectable, versionable, and owned by the product team	Requires engineering and can overfit the workflow if changed from weak evidence
Model weights	Stable, repeated behaviours with enough clean labelled data	Removes repeated prompt overhead and can generalise across tasks	Slowest and most expensive to change, hardest to inspect, and easiest to contaminate with poor labels

Default to the most reversible layer that can express the lesson. Put fresh knowledge in context. Put reliable workflow constraints and checks in the harness. Fine-tune only when the behaviour is stable, frequent, measurable, and supported by enough curated examples.

The harness is usually the strongest moat because it is the layer the product team fully controls. It can decide which signals count, how lessons are scoped, what must be evaluated, and who may approve a change. Better model weights cannot compensate for a loop that collects ambiguous labels or promotes regressions.

The governed learning loop

Self-learning is a production change process, not automatic prompt rewriting. Run it as a controlled cycle:

1. Capture. Record agent traces and meaningful user events with shared identifiers.
2. Join. Reconstruct the miss, the correction, and the verified outcome as one interaction record.
3. Classify. Decide whether the lesson is a fact, case, rule, tool defect, prompt defect, or workflow defect.
4. Propose. Let an analysis agent suggest a context, harness, or model change, but never treat its own diagnosis as proof.
5. Evaluate. Add the interaction to a private regression corpus and run the proposed change against the current baseline using the Evaluation Harness.
6. Approve and promote. Apply the change through the same review, versioning, security, and rollback controls as any other production change.
7. Monitor. Measure task success, human correction rate, abstention, cost, latency, and regressions after release.

This is how use becomes improvement without turning every user action into an unsafe self-modification. The agent may discover the lesson, but the evidence, evaluation gate, and deployment controls decide whether the system learns it.

Start with the capture boundary

Instrument one workflow where the agent proposes work and the human commonly corrects it. Capture both streams, define one verified outcome, and turn the first ten meaningful corrections into evaluation cases. Do not begin with fine-tuning. Begin by proving that the product can preserve a lesson and demonstrate that applying it improves the next run.

Related chapters

• The Evaluation Harness: proves that a proposed lesson improves the system before promotion.
• Execution Ledger: stores agent activity and the joined interaction record.
• Context Development Lifecycle: turns observed lessons into evaluated context changes.
• The Harness: the main product-owned surface for workflow improvement.
• Durable Agent Execution: preserves the traces and interaction state that this learning loop consumes.
• AG-UI: a technology note on one application-to-agent event protocol.
• Minimal Bootstrap: supplies the triggers, experimentation, and governed self-modification loop.
• Controls & Autonomy: constrains who or what may promote a learning change.