Turn Agent Traces Into Explanations With Attribution Cases

When an agent fails after many steps, a trace can be overwhelming.

It may show the request path, model calls, tool spans, retries, handoffs, and timestamps. That is valuable evidence. But it is not an explanation by itself.

A trace answers questions such as “what ran?”, “what overlapped?”, and “where did the request spend time?” It does not automatically answer “which observation, decision, dependency, or tool outcome caused the unacceptable result?” A confident model-generated summary is not proof either. It can be a useful hypothesis, but it must be tested against observable evidence.

For long-horizon agents, the unit of diagnosis should be an attribution case, not a giant transcript.

Use Trace Evidence to Test a Verdict

The OpenTelemetry trace model describes a trace as the path of a request through a system, composed of correlated spans. It can show hierarchy and timing, and span links can encode an instrumented association between related asynchronous operations. That makes a trace an excellent starting point for reconstruction, but the association alone does not prove which component caused an acceptance failure.

But correlation and ordering are not causal proof. A tool error may occur after the actual mistake. A retrieved document may be present in context but never influence a relevant decision. A retry may be a symptom of a missing prerequisite rather than the cause of the final failure. A span that looks central because it took the most time may be irrelevant to acceptance quality.

The difference matters because the remedy follows the attribution. If a team blames the most visible span, it may tune the wrong prompt, replace the wrong tool, or add a retry that hides the actual prerequisite failure.

Long-horizon work makes attribution harder

The Long-Horizon Agent Trajectory Attribution preprint frames the problem across trajectories that can include instructions, tools, observations, and memory. Its evaluation focuses on localizing contributing components and recovering causal chains. It is a recent preprint, not a general production guarantee, but its framing is useful: an agent failure can be distributed across a sequence rather than located at the final action.

The Long-Horizon Task Mirage preprint introduces HORIZON as a cross-domain diagnostic benchmark and treats failure analysis for long-horizon agents as a systematic problem rather than a search for a single embarrassing line in a log. Its conclusions are limited to its research setting. The transferable lesson is that diagnosis needs a method for narrowing the candidate set and testing explanations.

That method should preserve uncertainty. Many production failures have several plausible contributors: a stale observation, an underspecified goal, a changed tool response, an unavailable dependency, a poor decomposition, or an acceptance rule that was never made explicit. The correct outcome of an investigation can be “not yet established.”

The Failure Attribution Case

The Failure Attribution Case is a proposed record for a single diagnosis. It is intentionally smaller than a full trace and more disciplined than a post-hoc story.

• Case element: Task-level symptom — Question it answers: What accepted outcome, safety condition, or user need was not met? — Why it matters: Keeps the investigation anchored to a real failure instead of an unusual internal event.
• Case element: Scope and time boundary — Question it answers: Which task state, run window, and observable components are in scope? — Why it matters: Prevents an investigation from absorbing unrelated history.
• Case element: Candidate contributors — Question it answers: Which instruction category, observation, tool outcome, dependency, plan, or handoff could matter? — Why it matters: Makes alternatives explicit rather than hiding them behind one narrative.
• Case element: Evidence map — Question it answers: What trace references, artifact checks, state changes, and outcome observations support or weaken each candidate? — Why it matters: Separates retained evidence from inference.
• Case element: Dependency or causal chain — Question it answers: How could a candidate plausibly lead to the symptom? — Why it matters: Requires a mechanism, not merely temporal proximity.
• Case element: Test or counterfactual — Question it answers: What bounded replay, comparison, validation, or controlled change could distinguish the leading explanation? — Why it matters: Turns a story into something falsifiable.
• Case element: Confidence and alternatives — Question it answers: What is known, uncertain, and still plausible? — Why it matters: Prevents overclaiming from incomplete observability.
• Case element: Remediation and regression check — Question it answers: What limited change follows, and what future test would show it helped? — Why it matters: Connects diagnosis to a reversible engineering action.
• Case element: Privacy and retention boundary — Question it answers: Which summaries and references can be retained safely? — Why it matters: Avoids turning incident analysis into a collection of raw prompts, personal data, or hidden reasoning.

The case should refer to safe categories and evidence references where possible. It does not need to reproduce private instructions, credentials, customer data, raw tool payloads, or a model's hidden reasoning to be useful.

Start from the failed acceptance condition

The first line of an attribution case is not “the model made a bad decision.” It is a task-level symptom that can be inspected.

For example:

• the requested artifact did not meet a stated acceptance condition;
• an important claim lacked the required evidence;
• a relevant dependency changed and the final output was not revalidated;
• the agent stopped before a required review boundary; or
• a user-facing action was proposed without the necessary confirmation.

This framing helps teams avoid treating a model, tool, or person as the default culprit. It asks what failed in the observable contract first. Only then should the investigation enumerate plausible contributors.

Build hypotheses that can lose

A useful attribution hypothesis has a specific form:

If this candidate component had been different, the failure would probably not have occurred under the same relevant conditions.

That claim can be weak or strong depending on the evidence, but it can be challenged. A team might compare a run with the same task but a corrected prerequisite, replay a bounded stage with an updated observation, inspect whether a tool result was actually consumed, or check whether an acceptance rule was evaluated at all.

The point is not to recreate every historic action or assume a replay will be deterministic. The point is to choose the smallest safe test that distinguishes explanations.

• Weak diagnosis: “The tool failed.” — Stronger attribution case: “The agent acted on an outdated observation after the tool result had changed; a bounded revalidation test determines whether refreshing that observation resolves the acceptance failure.”
• Weak diagnosis: “The model ignored the instructions.” — Stronger attribution case: “The required acceptance condition was not present at the decision boundary; compare an explicit condition check with the same task shape before attributing the outcome to model behavior.”
• Weak diagnosis: “Parallel workers confused each other.” — Stronger attribution case: “Two branches produced incompatible artifacts without an explicit merge criterion; test a bounded integration check before changing the worker model or count.”

None of these examples assumes a private trace or names a person. Each names a candidate mechanism, evidence needed, and a test that can disconfirm it.

Use observability to narrow, then validate

Observability is still essential. The OpenTelemetry trace guidance shows how spans and links can establish request structure and related asynchronous work. A practical investigation can use that structure to narrow scope:

1. identify the failed acceptance condition and the smallest task boundary;
2. find the relevant span sequence, linked work, and observable state transitions;
3. list only the components that could plausibly alter the outcome;
4. record evidence for and against each candidate;
5. run or design a bounded validation; and
6. retain the leading explanation with its confidence and alternatives.

The trace is doing important work here: it prevents the team from guessing blindly. It does not grant permission to collapse correlation into causation.

Treat explanations as reviewable artifacts

An attribution case becomes more useful when another engineer can review it without reading an enormous sensitive transcript. That reviewer should be able to see:

• what failed and why it mattered;
• which components were considered;
• why one mechanism is more plausible than another;
• what evidence was directly observed versus inferred;
• what test was performed or remains necessary; and
• what narrow remediation will be checked next.

If the case cannot make those distinctions, it is a retrospective narrative, not yet an attribution record.

This complements causal task history. A causal history captures which work and decisions depend on each other so a system can scope a review or revalidation. Failure attribution asks a later, sharper question: which candidate contributor is supported by evidence strongly enough to change the system? Both are useful, but neither replaces the other.

A practical adoption path

1. Choose one recent failure with a clear acceptance condition and a bounded task history.
2. Write the symptom in user- or task-level terms, not as a presumed model defect.
3. Assemble only the relevant trace references, linked work, artifact checks, and state observations.
4. List two or more plausible contributors before selecting a leading hypothesis.
5. State the mechanism by which each candidate could affect the symptom.
6. Design the smallest safe comparison, replay, or validation that could weaken the leading explanation.
7. Record confidence, remaining alternatives, a narrow remediation, and a regression check.
8. Retain a privacy-conscious summary and the safe evidence references, not a broad dump of prompts, raw results, or hidden reasoning.

Over time, teams will build a library of tested failure patterns. That library should guide new evaluations and safeguards. It should not become a blame database or a substitute for re-checking the evidence in a new context.

A Virillio Code Editorial Perspective

Virillio Code Editorial uses this topic to articulate a general lesson: an agent failure needs evidence explaining why a component is implicated instead of relying on a long record of events. The Failure Attribution Case is a proposed educational framework, not a description of a shipped or planned Virillio Code capability.

Sources and further reading

• Long-Horizon Agent Trajectory Attribution — An August 2026 preprint formalizes attribution across long-horizon trajectories with instructions, tools, observations, and memory, and studies localization and causal-chain recovery.
• The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break — A clearly labeled April 2026 preprint introduces the HORIZON cross-domain diagnostic benchmark for systematic analysis of long-horizon agent failures.
• Traces — OpenTelemetry — Official documentation defines traces and spans, including hierarchy, correlation, and span links that associate related asynchronous work without independently proving root cause.

Editorial disclosure

This article was substantially researched, drafted, and revised with AI through the Virillio Code editorial workflow. Virillio Code publishes the final text under its editorial byline, and the supporting primary sources are linked above.