A Coding-Agent Debugging Trace Needs a Hypothesis Ledger

A coding agent sees a failure, reads a few files, runs a check, edits code, and repeats. The resulting trace can be very long.

Length is not diagnosis.

The trace may show what the agent did. It rarely tells a reviewer which explanation it was testing, what result would have disproved that explanation, or why the next change was justified instead of merely plausible.

That distinction matters most when the first explanation is wrong. A failing check can reflect a regression, an unsupported setup, a stale report, a dependency change, a partial prior fix, an environment mismatch, or an observation that does not isolate the reported behavior. A patch made before those possibilities are separated can hide the symptom while increasing future uncertainty.

A coding-agent debugging trace needs a Hypothesis Ledger.

Tools Need a Diagnostic Structure

The Debug2Fix preprint argues that coding agents often rely on static inspection or iterative test-and-fix cycles that require them to infer runtime state from limited signals. It evaluates a framework that makes interactive debugging a more explicit part of the agent workflow. The paper’s performance results apply to its evaluated models and benchmarks, not every coding environment. The broader design lesson is still useful: an agent needs evidence that can discriminate among explanations, not simply more opportunities to edit.

The DDBench preprint makes a complementary point in distributed-system repair. It evaluates symptom-only tasks against tasks with bounded debugging context such as logs, traces, runtime state, and investigation notes. The authors report that curated context can improve results in their 60-bug benchmark, but also state that even faithful context can mislead. Context is an input to diagnosis, not a verdict.

The correct response is not to preserve every log line or expose every internal trace to a reviewer. It is to preserve the small set of diagnostic commitments that make the investigation challengeable.

The Hypothesis Ledger

The Hypothesis Ledger is a proposed application-level record for a live investigation. It does not claim that an agent has found the root cause. It makes the difference between an observation, a candidate explanation, and a justified next check visible.

• Ledger element: Symptom and boundary — Question it answers: What behavior was observed, under which stated conditions? — Why it matters: Separates the report from an inferred cause.
• Ledger element: Candidate explanations — Question it answers: Which plausible mechanisms could account for the symptom? — Why it matters: Prevents the first plausible story from becoming fact.
• Ledger element: Current evidence — Question it answers: What supports or challenges each explanation? — Why it matters: Gives the agent a bounded basis for its next move.
• Ledger element: Discriminating check — Question it answers: What permitted observation would make the alternatives meaningfully diverge? — Why it matters: Moves the work from trial-and-error toward testable investigation.
• Ledger element: Predicted outcomes — Question it answers: What would each result imply, and what would it not imply? — Why it matters: Makes confirmation bias easier to spot.
• Ledger element: Observation quality — Question it answers: Was the check complete, reproducible enough, partial, flaky, or blocked? — Why it matters: Keeps an ambiguous result from being treated as decisive evidence.
• Ledger element: Revision and disposition — Question it answers: Which explanation remains viable, which is weakened, and what work may proceed? — Why it matters: Preserves a clear diagnostic state instead of a stream of edits.
• Ledger element: Stop or escalation condition — Question it answers: When should the agent hold, ask, or seek authorized review? — Why it matters: Treats uncertainty and inaction as valid outcomes.

The ledger should not store raw credentials, private URLs, hidden prompts, complete source payloads, or unrestricted logs. A concise description of the approved evidence boundary is more useful and safer for most review workflows.

A trace and a ledger answer different questions

An agent failure attribution case is useful after a failure: it asks whether the record supports assigning a cause to a component, policy, input, tool, or environment. The Hypothesis Ledger is earlier and narrower. It asks:

Before the agent changes code or assigns blame, which explanation is it trying to distinguish, and what result would change its mind?

That difference has practical consequences.

• Record: Activity trace — Main question: What events happened? — Appropriate timing: Throughout a run
• Record: Hypothesis Ledger — Main question: What candidate explanation is being tested, and what result would revise it? — Appropriate timing: During investigation
• Record: Attribution case — Main question: What does the completed evidence support about a failure’s cause? — Appropriate timing: After sufficient evidence exists
• Record: Review handoff — Main question: What changed, what was checked, and what remains uncertain? — Appropriate timing: When a change is ready for review

No one record replaces the others. The ledger prevents an activity trace from being mistaken for proof that the agent’s reasoning was sound.

An illustrative diagnosis

Consider a report that a background task repeats unexpectedly. The agent should begin with the symptom, not a patch:

• Candidate explanation: The task is classified as retryable after a handled failure. — Bounded discriminating check: Inspect the approved status and classification evidence for a representative occurrence. — Result interpretation: A matching classification supports further investigation of the classification rule; it does not prove the underlying failure is expected.
• Candidate explanation: A scheduling condition continues to admit identical work. — Bounded discriminating check: Compare the task identity and admission evidence for two occurrences within the permitted scope. — Result interpretation: Repeated admission evidence weakens the “retry only” explanation.
• Candidate explanation: The report describes behavior already resolved or only partly reproduced. — Bounded discriminating check: Reproduce the stated symptom in a bounded, authorized check, or record that it cannot be reproduced. — Result interpretation: A non-reproduction is a finding, not permission to invent a fix.

The example deliberately stops short of commands, private identifiers, or a production remediation. It shows the investigative shape: each next action has a question, a boundary, and a possible reason to stop.

Demand predictions before edits

A weak loop looks like this:

1. Read a symptom.
2. Form a plausible theory.
3. Edit code.
4. Run a check.
5. Explain the result as support for the theory.

The pattern is dangerous because a changed test outcome may be compatible with several theories. A patch can even hide the evidence needed to understand the original behavior.

A stronger loop asks the agent to write its prediction first:

1. State the observed symptom and the candidate explanations.
2. Choose the smallest permitted check that makes the explanations differ.
3. State what each possible result would support, weaken, or leave unresolved.
4. Perform the check and record its quality.
5. Revise the candidates before deciding whether a code change is justified.

This is not a demand for formal proof. Debugging often cannot produce it. It is a demand for falsifiability: the agent should be able to say what would make it change course.

Treat no change as a legitimate disposition

The FixedBench preprint evaluates 200 human-verified reports for which no code change is required and finds that coding agents can propose undesirable modifications instead of abstaining. Its result is limited to its benchmark and studied systems, but it highlights a healthy workflow rule: a diagnosis can end with “no change justified,” “insufficient evidence,” or “needs a decision.”

The Hypothesis Ledger should therefore include explicit outcomes:

• Disposition: Supported for a bounded change — Meaning: Evidence supports proceeding within the recorded scope, subject to ordinary review and verification.
• Disposition: Needs more evidence — Meaning: The candidate explanations remain insufficiently separated.
• Disposition: Contradicted — Meaning: The result weakens the leading explanation; revise the investigation instead of defending it.
• Disposition: Already resolved or not reproduced — Meaning: Do not add a speculative patch; report the evidence boundary and any remaining uncertainty.
• Disposition: Held or escalated — Meaning: A missing decision, authority, environment, or safety boundary prevents further responsible investigation.

This is how an agent stops treating movement as progress.

Measure Diagnostic Quality Alongside Fix Rate

Teams can evaluate the ledger without expecting every investigation to finish:

• Discriminating-check rate: What share of investigations contain a check that could meaningfully distinguish candidate explanations?
• Premature-change rate: How often does the first code change precede a recorded symptom boundary or discriminating check?
• Revision honesty: When evidence contradicts an early theory, does the agent revise it or merely add a new narrative?
• No-change precision: How often is a held, contradicted, or already-resolved disposition later shown to be appropriate?
• Evidence minimization: Does the record preserve enough to review the decision without collecting private or irrelevant material?

These are not universal metrics. They are prompts for teams to ask whether an agent has become a better investigator, rather than merely a faster editor.

A practical adoption path

1. Start with one recurring class of coding-agent investigation, such as test failures or reported regressions.
2. Require a symptom boundary and at least two plausible explanations before any material change.
3. Ask the agent to name the smallest permitted discriminating check and its predicted outcomes.
4. Require a revision entry after each meaningful observation.
5. Let the task end in held, contradicted, or no-change dispositions when the evidence warrants them.
6. Use later review outcomes to improve the quality of the questions and checks, not to penalize honest uncertainty.

The aim is not to turn debugging into paperwork. It is to give a reviewer one compact answer to the most important question: why did the agent believe this next change was the right experiment?

A general lesson for coding agents

For a coding agent, an execution trace is a history of activity. A Hypothesis Ledger is a history of testable diagnostic commitments. The ledger is a proposed educational framework, not a description of a released Virillio Code capability.

Sources and further reading

• Debug2Fix: Can Interactive Debugging Help Coding Agents Fix More Bugs? — A preprint contrasts static and iterative test-fix behavior with interactive debugging and reports benchmark-specific gains from its debugging-focused framework.
• Evaluating Agentic Code Repair Capabilities in Distributed Systems — The DDBench preprint studies 60 bugs from 13 open-source distributed systems and reports that bounded debugging context can help while also misleading agents when curation is poor.
• Coding Agents Don’t Know When to Act — The FixedBench preprint studies 200 human-verified reports for which no code change is required and argues that inaction should be treated as a success path.

Editorial disclosure

This article was substantially researched, drafted, and revised with AI through the Virillio Code editorial workflow. Virillio Code publishes the final text under its editorial byline, and the supporting primary sources are linked above.