Summarization for Long-Running Agents: Compression Without Losing the Plot

🧠 Summarization Is State Compression, Not Cosmetic Editing

The key distinction is simple: a human summary aims to tell a coherent story, but an agent summary must preserve decision-critical state. Those are not the same goal. A readable paragraph often smooths over uncertainty, removes exact identifiers, and compresses failed attempts into vague phrases like “investigated several options.” For a human status update, that may be acceptable. For an agent that has to decide what to do next, it is a direct source of error.

That is why compression quality affects behavior, not just cost. Research on governance decay shows that when constraints are lost during compaction, previously compliant systems can begin violating rules once the original text is no longer visible. The failure is subtle because the model still appears fluent. The degradation happens in the state it is reasoning from, not in the grammar of the output. A useful reference on that risk is this paper.

For AI agents for developers, this framing matters because the hot path is operational. The agent is choosing tools, filling arguments, interpreting logs, and deciding whether to continue or stop. If compression preserves the wrong things, the loop mutates. Developers often think of summary quality as a prompting concern. In practice, it behaves more like agent state management inside an LLM agent architecture. You are maintaining a lossy replica of execution state, and the reliability of the whole workflow depends on how carefully that replica is built.

State compression means you need to think about what the next step actually requires. A good compacted memory is not a shorter version of everything. It is a selective representation of the facts that still constrain future action. That includes goals, prohibitions, unresolved blockers, and evidence that invalidated prior options. If the agent learned that package version 2.4.1 breaks tests but 2.3.9 passes, the summary should not merely say “dependency issue investigated.” It should retain the specific versions because future tool calls may depend on them.

This is why “better writing” is not the goal. In fact, prose that reads better to a human can be worse for an agent. Humans infer missing detail from context. Models do not recover omitted state reliably, especially after many turns and tool interactions. When the summary says “the previous workaround failed,” you may know which workaround is meant because you read the trace. The model may not. Compression has to serve retrieval and decision-making first. Readability is helpful, but only after fidelity is protected.

🧩 Why Raw History Becomes Adversarial Context

There is a point in long-running agents where more transcript actively hurts. Not because old information is always irrelevant, but because relevant and irrelevant details become entangled. A coding agent might accumulate package install logs, stack traces, test output, file diffs, retries, and partial reasoning. The signal is in there somewhere, but the next model call has to pick it out from a mountain of residue.

This is one reason bounded tool output matters before summarization even begins. OpenAI’s coding agent guidance describes capping large shell outputs while preserving the beginning and end with truncation markers. That is not a cosmetic feature. It is observation compression. You reduce noisy evidence before it pollutes the conversational state. If you skip that step, the later summary is forced to compress garbage mixed with the facts that actually matter. That makes downstream compaction less trustworthy.

The practical consequence is that long context windows can hide poor discipline. Teams assume bigger windows remove the need for memory design, but the real issue is relevance density. A 100,000 token history full of stale logs can be worse than a 10,000 token working set with explicit constraints, decisions, blockers, and artifact pointers. The lesson is slightly uncomfortable but useful: not all context is help. In long-running agent state management, some context is drag.

It helps to think of raw history as unindexed storage. Everything may still be present, but presence is not the same as usability. If a transcript contains three failed API attempts, two corrected assumptions, a final user instruction, and fifty screens of logs, the agent has to re-derive the active state each time it acts. That repeated reconstruction is expensive and brittle. The more cluttered the history becomes, the easier it is for salience to drift toward the wrong details.

There is also a behavioral effect. Old traces do not merely consume tokens. They influence what the model attends to. Repetitive logs, repeated warnings, or earlier abandoned plans can keep pulling the model back toward stale branches. That is why raw history can become adversarial context. It does not attack the system intentionally, but it competes with the true current state for attention. If you have ever watched an agent re-open a path that was already ruled out, this is often the mechanism behind it.

For developers, this matters because the failure mode looks like bad reasoning when it is often bad state presentation. The model may be perfectly capable of making the right choice if shown the relevant facts clearly. What breaks is the packaging of those facts. That is an encouraging insight because it means the fix is not always “use a smarter model.” Sometimes the fix is simply to reduce noise, bound observations, and compact history into something the model can reason over consistently. This is especially relevant for tool calling agents and AI coding agents that live inside a fast-moving agent loop.

⚙️ There Are Three Different Compression Jobs

A lot of confusion disappears once you separate three layers of compression. First, observation compression trims noisy tool output. Second, conversation compaction summarizes older turns. Third, state extraction writes durable facts or decisions into external systems such as files, databases, tickets, or memory stores. These layers solve different problems, so treating them as one thing makes debugging harder.

Observation compression protects the agent from verbosity at the source. Think shell output, huge JSON payloads, or repetitive logs. Conversation compaction protects the working context window by replacing old turns with a shorter state representation. State extraction protects durability by moving key facts into systems that are not lossy by design. Anthropic’s guidance on long-running agents is especially good on this point: leave clear artifacts for future sessions instead of relying on prose memory alone. See their approach here.

Why this matters in real workflows is straightforward. If a Jira ticket is the source of truth, do not ask a rolling summary to remember the ticket status perfectly forever. If a file path or deployment ID is critical, store it durably and let summaries point to it. Hot-path summaries should carry operational state for the next reasoning step, not impersonate a database. That separation keeps compression useful without pretending it is lossless.

These three jobs also fail differently. If observation compression is weak, the agent drowns in noise before higher-level summarization even starts. If conversation compaction is weak, the agent forgets which choices were made and why. If state extraction is weak, the system cannot recover when the summary drops a critical detail. Keeping the layers distinct gives you a cleaner way to debug. When an agent repeats a failed command, ask whether the failure came from bloated logs, a missing decision in the summary, or the absence of a durable artifact that should have existed outside the summary.

Consider a bug-fixing agent working through a long incident. Observation compression might reduce a 20,000 line stack trace to the exception type, top frames, and a pointer to the full log file. Conversation compaction might preserve that “patch A failed because migration order was wrong” and “customer data cannot be touched.” State extraction might write the final reproduction steps and affected commit hashes into a ticket or markdown file. Each layer has a different role, and the system becomes much easier to reason about when each role is explicit.

This separation also maps cleanly to short-term vs long-term memory in agents. The compacted conversational state supports short-term reasoning in the active loop, while durable artifacts and memory stores support long-term recall, procedural memory for agents, and safer agent handoffs across sessions or systems.

🗂️ A Good Agent Summary Looks More Like a Schema Than a Paragraph

The safest design pattern is a structured summary. Free-form prose invites omission because the model optimizes for narrative flow. A schema creates a checklist. It forces the summarizer to represent categories that later decisions depend on, even if those categories are awkward to mention in a natural paragraph.

For long-running agents, a practical schema usually includes current objective, immutable constraints, decisions already made, open issues, unresolved errors, artifact references, and next expected action. That is enough to preserve both direction and caution. It also keeps negative state visible. “Do not retry before 14:05 UTC” and “endpoint X ruled out after 2x 429” are not decorative details. They are control data.

{
"objective": "fix failing OAuth refresh flow",
"constraints": ["do not modify prod", "no retries before 14:05 UTC"],
"decisions": ["ruled out endpoint X after 2x 429"],
"open_issues": ["staging token refresh still failing"],
"artifacts": ["ticket AUTH-431", "logs/auth_refresh.txt"],
"next_action": "test cached fallback in staging"
}

This format works because the next step can query state by field. Tool-calling agents can inspect constraints before choosing a tool, inspect decisions to avoid repeating failed paths, inspect open issues to avoid false completion, and inspect artifacts to fetch source material when details are missing. Research on memory structures keeps converging on this pattern because it reduces silent omission. In practice, structure is not bureaucracy. It is error prevention.

Each field exists for a reason. The objective tells the model what “progress” means, which sounds obvious until an agent begins optimizing for activity rather than completion. Constraints protect boundaries that must survive every compaction event. Decisions preserve what has already been learned so the system does not keep paying for the same mistake. Open issues stop the agent from declaring victory too early. Artifacts bridge compressed memory with external evidence. Next action keeps the loop moving in a concrete direction instead of falling back into broad re-analysis.

A schema also makes it easier to compare summaries over time. If one checkpoint includes a staging-only restriction and the next checkpoint does not, that change is visible immediately. In free-form prose, the same omission may be hidden inside a paragraph that still sounds plausible. Developers benefit from this because production problems are easier to isolate when memory has structure. You can diff fields, validate slots, and trace exactly when key state changed.

Another advantage is interoperability. Structured summaries are easier to feed into tool selectors, validators, and agent orchestration logic. A tool router can read the constraints field before allowing a deployment action. A recovery routine can check open_issues before marking a workflow done. A monitoring system can alert when next_action remains unchanged across multiple loops, which is often a sign of a stuck agent. The schema is not only for the model. It is also for the software around the model.

🚫 The Details Most Summaries Lose First

Compression failures are rarely random. The first things to disappear are usually the details that matter most operationally. Exact IDs, filenames, environment names, timestamps, parameter values, and unresolved questions are easy to paraphrase away. A summary may say “the relevant logs were saved” while losing the path logs/auth_refresh.txt . It may say “rate limiting occurred” while dropping the wait-until time that should gate the next retry.

Another common failure is that summarizers reward closure. They tend to write “integration mostly complete” because that sounds coherent, even when the live state is “OAuth refresh works locally but still fails in staging.” That missing clause changes everything. The agent shifts from debugging mode to wrap-up mode. The task appears done when it is not.

Negative decisions are especially fragile. Research has started calling this out directly: compressed context often preserves what was done but loses what was ruled out or prohibited. That biases successor behavior toward action when restraint was the correct outcome. If an agent previously decided not to run a destructive operation because of policy, and that non-action disappears from memory, the next pass may rediscover the same path and take it. For tool-using systems, this is not a minor formatting issue. It is a safety issue and a reliability issue at the same time.

Why do these details disappear first? Because they are awkward. They do not sound elegant in prose, and they often look secondary to the larger narrative. But operationally, they are often the larger narrative. A single environment name can be the difference between a harmless test and a production incident. A single timestamp can determine whether a retry is valid or guaranteed to fail. A single file path can be the only link back to the evidence that explains a prior decision.

There is also a bias toward summarizing positive progress rather than preserving unresolved tension. Human writers naturally compress toward “what happened.” Agents need “what still constrains what can happen next.” That is why a vague sentence like “API issues investigated” is less useful than “endpoint X rejected after two 429 responses, use fallback Y only in staging.” The second form is slightly uglier to read, but it is vastly more actionable.

If you want a practical rule, watch anything that could appear in a command, an API argument, or a policy check. IDs, branch names, file paths, environment labels, time gates, blocked options, and pending blockers all deserve special protection. These details matter because they are the interface between memory and action. Once they vanish, the agent may still appear informed while actually operating on guesswork.

⏱️ Trigger Policies Matter as Much as Summary Quality

Summarization should not happen randomly or only when something crashes. It needs explicit trigger rules. LangChain’s built-in middleware makes this concrete with threshold-based compaction that can trigger on token count, message count, or fraction of context window, while also preserving a recent verbatim tail. Their middleware docs are useful because they frame summarization as engineering control, not magical prompt phrasing. See the middleware reference.

from langchain.agents.middleware import SummarizationMiddleware
middleware = SummarizationMiddleware(
model="gpt-5.4-mini",
trigger=("tokens", 4000),
keep=("messages", 20)
)

The important mechanism is not the exact model name. It is the split between what gets compacted and what remains raw. Keeping the last several turns verbatim preserves local grounding, including fresh tool results and user corrections, while the summary carries older global state. That hybrid design is usually stronger than replacing everything with one rolling paragraph.

Why this matters in production is subtle. Triggering only on token count can miss long loops made of many short turns. Triggering only on message count can compact too early when messages are tiny but still important. OR and AND logic across thresholds helps catch both patterns. For long-running agents, compaction timing is part of agent observability and reliability. Summarize too late and you drown in noise. Summarize too early and you erase context that has not settled yet.

A useful way to design triggers is to match them to workflow rhythm. Tool-heavy coding agents often need compaction after large outputs or after a sequence of retries. Research assistants may need it after topic shifts or after a bundle of extracted sources accumulates. Customer support agents may need it at escalation boundaries, where stable facts should be preserved before new participants enter the thread. A single threshold for every workload is convenient, but it usually ignores the shape of the actual task.

The preserved verbatim tail matters for another reason: recency often carries correction. Users amend instructions. Tools return the latest evidence. The agent may have just discovered that a prior assumption was wrong. If you compact that fresh material too aggressively, the summary may freeze the older interpretation before the correction has had time to propagate. Keeping a local raw window lowers that risk by giving the next step access to the newest facts exactly as they appeared.

In mature systems, trigger policy becomes part of reliability engineering. You monitor how often compaction occurs, what average compression ratio you get, and whether agent behavior changes after summary events. If failures cluster right after compaction, the problem may not be the core model at all. It may be that your triggers fire at the wrong moments, or compact the wrong span, or leave too little recent context untouched. That is why trigger policy deserves design attention instead of being treated like a default middleware setting. This is also where LangGraph memory and similar memory runtimes become operational rather than conceptual.

🔍 How to Detect Summary Drift Without Fooling Yourself

The wrong evaluation question is “Does this summary sound accurate?” That mostly measures readability. The useful question is whether the agent behaves the same with full history and with summarized history. This is a much stricter standard, and it maps directly to how agent loops actually fail.

A practical drift check is to replay the same next-step decision twice. First, give the model the full history. Second, give it the summarized state plus whatever recent verbatim context your runtime would normally keep. Then compare tool choice, argument accuracy, constraint adherence, and recognition of unresolved blockers. If the full-history version waits until 14:05 UTC but the summarized version retries immediately, the summary failed, even if it reads beautifully.

This framing aligns well with modern tracing and eval practice for AI agents for developers. You are not grading prose; you are grading state fidelity under compression. That makes summary quality measurable. It also makes regressions visible when a summarizer prompt, middleware threshold, or model version changes. For teams already using traces and agent-level evals, summary drift should be treated as its own test target inside the broader agent evaluation and tracing workflow.

There are several concrete dimensions to compare. Tool choice tells you whether the summary changed the agent’s view of what category of action is appropriate. Argument accuracy tells you whether important entities survived compression, such as filenames, IDs, or environment names. Constraint adherence tells you whether rules and prohibitions stayed visible. Blocker recognition tells you whether the agent still understands the task as incomplete. Together these checks cover most of the practical ways memory drift shows up in production.

This matters because drift is often silent. The summarized version may still produce a plausible explanation for its action. It may even sound more confident than the full-history version because the compressed state removed the uncertainty that should have remained. Without behavior-based comparison, teams can mistake smoothness for correctness. A summary that reads cleanly but changes the next action is not a better summary. It is a misleading one.

If you already collect traces, use them. Pick a handful of real incidents where the correct next step is well understood. Re-run them with full and compacted memory, then inspect the divergence. Over time, this gives you a practical regression suite for memory quality. The deeper lesson is that summary drift is not a literary issue. It is a systems issue, and systems should be tested by outcomes. That is true whether you are building Responses API agents, using the OpenAI Agents SDK, or stitching together a broader multi-agent system.

🧪 A Concrete Validator Pattern for Structured Summaries

One useful pattern is to treat summary generation as a controlled transform with validation. Instead of asking the model to “summarize so far,” ask it to fill explicit fields, then reject outputs that omit required slots. This does not eliminate drift, but it catches shallow failures before they enter the hot path.

required_fields = [
"objective",
"constraints",
"decisions",
"open_issues",
"artifacts",
"next_action"
]
summary = generate_summary(state)
for field in required_fields:
if field not in summary:
raise ValueError(f"Missing summary field: {field}")
if not summary["constraints"]:
raise ValueError("Constraints cannot be empty")
if not summary["open_issues"] and task_not_done(state):
raise ValueError("Open issues missing while task remains incomplete")

The point is not strict formalism for its own sake. The point is forcing the system to declare whether constraints still exist, whether blockers remain, and what the next action is supposed to be. In high-stakes workflows, you can add human review or stronger checks on exact entities like ticket IDs, filenames, or environment names.

This matters because long-running agents are already a stateful system. Once you accept that, validation becomes obvious. We validate tool arguments, API payloads, and database writes. Summaries should get similar treatment because they influence the same downstream behavior. If you let lossy compressed state flow through the loop without checks, you are accepting silent corruption as normal.

The logic in the validator is simple, but each part protects against a specific failure mode. The required field check catches omission at the structural level. If the summary has no constraints field at all, there is no point pretending the result is usable. The non-empty constraints rule catches a more subtle failure where the field exists but has been emptied during compaction. The open-issues check protects against false closure by ensuring the model cannot quietly convert an unfinished task into a finished-looking memory state.

You can go further with semantic validation. Compare named entities in the summary against entities extracted from recent turns. If the trace mentions AUTH-431 , staging , and logs/auth_refresh.txt , the validator can require those entities to appear somewhere appropriate. You can also enforce type rules, such as “constraints must be a list of explicit prohibitions or limits” rather than vague text. None of this makes summarization perfect, but it moves failure from invisible runtime drift to explicit validation errors.

Why this matters for long-term system health is straightforward. Validators create pressure toward stable schemas and predictable memory behavior. That makes the surrounding system easier to evolve. When you swap summarizer models, tune prompts, or change compaction thresholds, the validator acts as a guardrail. Instead of discovering a memory regression weeks later through odd agent behavior, you catch it near the point where the bad state is generated.

🧭 Version Summaries and Keep Raw Evidence

A safe memory architecture is non-destructive. Keep raw logs, tool traces, and durable artifacts as the source of truth. Use summaries as hot-path working state, not as the only record. This fits both Anthropic’s artifact-based handoff style and LangChain’s separation between short-term thread state and other memory layers. Their short-term memory framing is a good reference here.

Versioning matters for the same reason. Instead of overwriting one rolling recap forever, keep timestamped summary checkpoints linked to raw trace ranges. If the agent suddenly starts violating a constraint or re-opening a path that was already ruled out, you need to know when that fact disappeared. Without checkpoints, summary regressions are hard to localize. With checkpoints, you can compare state before and after a compaction event and inspect exactly what was dropped.

This is why “external source of truth” is not a boring architectural note. It is your recovery path. Summaries are lossy by design. When they fail, you need somewhere authoritative to fetch the original evidence. In long-running agents, durability is not only about surviving crashes. It is also about surviving bad compression.

Think of summary versioning the way you think about database migrations or deployment artifacts. You would not overwrite production state without a trail. Agent memory deserves the same discipline because it can directly influence production behavior. A timestamped summary plus a pointer to the raw trace span gives you a precise audit path. When something goes wrong, you can answer two useful questions quickly: what did the agent believe at this moment, and where did that belief come from?

This becomes especially valuable in collaborative settings. One agent may hand work to another, or a human may re-enter the loop after several compaction cycles. A versioned summary history makes the handoff understandable. Instead of trusting one opaque rolling paragraph, the next participant can inspect how the state evolved and whether any critical constraint narrowed, expanded, or vanished over time. That is not just useful for debugging. It improves operational trust.

It also supports supervisor worker agents and other multi-agent systems where agent handoffs are routine. If one worker agent compresses state poorly, the supervising agent inherits a distorted view of the task. Versioned summaries plus durable artifacts make those handoffs more reliable and give your agent orchestration layer something concrete to inspect.

⚠️ When Bad Compression Changes Tool Behavior

One of the most practical reasons to care about summary quality is tool drift. Recent work shows that memory, including compressed memory, can pull attention toward superficially related but task-irrelevant details, degrading tool choice and parameter selection. That maps directly to the failures developers already see: wrong tool, wrong args, repeated retries, and premature stopping. A relevant study is here.

Imagine a coding agent debugging OAuth refresh in staging. The full history clearly shows that endpoint X is rate-limited, production must not be touched, and the open issue is isolated to staging token refresh. A weak rolling summary collapses this into “investigated endpoint issues, fallback pending.” Now the next step is underspecified. The agent may hit endpoint X again, forget the retry delay, or test the wrong environment. Nothing about the model’s reasoning style needs to change for this to happen. The state it saw was simply worse.

That is why the closing thought is worth taking literally: a bad summary is corrupted state. The system may still run and still produce plausible output, which makes the failure easy to miss. But under the hood, it is reasoning from a degraded replica of reality. For long-running agents, that is the real compression problem to solve.

The real-world impact is larger than one mistaken tool call. Tool behavior compounds. A wrong environment selection can generate misleading logs, which then enter the trace and distort later summaries. A dropped prohibition can trigger an action that creates side effects, forcing the system to recover from damage rather than continue the original task. Once memory corruption changes action, action creates new evidence, and that new evidence can reinforce the bad trajectory. This is one reason memory bugs are so expensive in agent loops: they are not static bugs. They can cascade.

The practical lesson is slightly sobering but useful. If you want reliable long-running agents, treat summarization with the same seriousness you give to tool permissions, retry logic, durable execution for agents, and state persistence. Memory compression sits on the path between observation and action. When it works, the agent stays focused and efficient. When it fails, the system can look intelligent while moving in the wrong direction. Understanding that distinction is what keeps compression from quietly losing the plot and improves overall agent reliability.