Deep Agents' Context Compaction: Offloading, Summarization, and the Art of Keeping History Alive

The Core Philosophy: Compaction Reduces, Not Erases

The central insight behind Deep Agents' context management is that a conversation summary is not a replacement for the work itself—it's a navigation aid. When I think about what compaction should do, I hold this principle: the model's active context is a working set, not an archive. The archive lives on disk, intact, queryable, and re-readable. The active context carries just enough to keep the agent oriented and productive.

This is a fundamentally different posture from naive truncation. A sliding-window approach that simply drops old turns when tokens run out has a clean elegance to it—it's cheap, predictable, and requires no storage. But it carries a hidden cost: everything that was evicted is gone. No amount of clever prompting or careful engineering can bring back a tool output that was deleted. The agent either remembers it (the model's own parametric memory, which is unreliable for exact details like line numbers or stack traces) or it does not.

Deep Agents rejects that trade. Instead, the pipeline does this:

1. Raw artifacts go to disk, not the void. Large tool outputs (web scrapes, file reads, command traces) are written to a sandboxed filesystem under paths like /conversation_history/. The full payload survives.
2. The active context gets a summary and a pointer. In place of the raw data, the model sees a short description like "Build failed with 14 TypeScript errors" plus a reference to the file where the full output lives.
3. The agent re-reads when needed. If the agent needs to know which lines failed, it issues a read_file or grep call against the offloaded artifact.

What makes this work is the separation of what the model needs to think from what the agent needs to survive. The model does not need a 500-line compiler trace in-context to reason about whether the fix it's about to apply is plausible. But the agent must have that trace available somewhere, because the actual work of fixing the code will require inspecting the exact error locations. Compaction makes the active context smaller without making the work history thinner.

sequenceDiagram
    participant Agent as Deep Agent (LLM)
    participant Tools as Tool Layer
    participant FS as Sandbox FS (/conversation_history/)
    participant Ctx as Active Context Window

    Agent->>Tools: Execute tool (grep, read_file, run_tests)
    Tools->>FS: Write raw output to /conversation_history/step_XX.log
    Tools->>Ctx: Return summary + file pointer
    Note over Ctx: Model reasons with compact summary
    Agent->>FS: Issue read_file to inspect full artifact
    FS->>Agent: Return exact content
    Agent->>Ctx: Reason with complete data (only when needed)

The failure mode I'd watch for in a naive implementation is treating the file pointer as decorative—a reference the model "knows about" but never actually pulls. That's a different problem, and it's a genuine hazard. But the architecture itself is sound because it preserves the choice. Truncation makes the choice for you; offloading keeps it in the model's hands. That distinction—between losing information and making it less immediately available—is the whole point.

Mechanics: How the Pipeline Actually Works

The Deep Agents approach uses two complementary mechanisms that work at different levels of granularity. The first is filesystem offloading, which handles the raw data problem. The second is middleware-driven summarization, which handles the token budget problem. They compose into a single pipeline that is both thinner and more durable than what a naive compaction strategy would produce.

Filesystem offloading is the foundation. When a tool returns a large payload—a full repository diff, a long test suite output, a web scrape—the harness detects that the output exceeds a size threshold and writes the raw content to a workspace directory, typically under /conversation_history/ or a similar artifact path. In the active context, the model sees a concise reference: "Full diff written to /conversation_history/step_03_diff.patch." The same treatment applies to large inputs. If the agent invokes a tool with a massive argument payload (say, an entire file's worth of edit instructions), that argument is offloaded to disk as well, with a placeholder in the message trace. This is important because tool call arguments are part of the conversation history and consume tokens just as outputs do.

Middleware-driven summarization is the second layer. Deep Agents uses a SummarizationMiddleware that monitors the context window. When token usage approaches a model-specific threshold—typically around 85% of the model's context limit—the middleware triggers a compression pass. This pass synthesizes older message turns into a structured summary block, preserving the recent turns (the last N messages, where N is configurable) intact. The summary is generated by an LLM call, meaning it's not a mechanical trim but a meaningful abstraction of what happened so far. The key design constraint here is the transactional turn boundary requirement: compaction only runs after a complete User → Assistant → Tool(s) → Assistant cycle resolves. Breaking a sequence with a pending tool call would cause API schema validation errors on many providers, especially when a ToolMessage needs to immediately follow its paired AIMessage(tool_calls=[...]).

The third element is the compact_conversation tool, exposed via create_summarization_tool_middleware. This gives the agent autonomous control over compaction. Instead of relying solely on an arbitrary token count trigger, the model can decide to compact at logical boundaries—after finishing a research phase before starting implementation, for example. The design insight here is that the model knows when it's about to change mental context better than a token counter does. A rigid trigger can fire mid-task during a fragile operation, disrupting the agent's flow. Agent-initiated compaction avoids that.

Underneath all this sits a state management architecture that is the real quiet hero. Deep Agents separates non-message state channels from the message list that gets compacted. Channels like async_subagent_jobs (which track in-flight subagent tasks), filesystem paths, and plan/scratchpad progress live in their own slots in the LangGraph state. When a compaction pass runs, it processes only the message history. It never touches the plan, the subagent job registry, or any other non-message state. This is what makes compaction safe: the summarization can be aggressive with conversation history without risking the loss of execution state.

Everything here is configurable. Token thresholds, the number of recent messages preserved, the summarization prompt, the target model that performs the compression, and the filesystem backend (local disk, Docker sandbox, remote storage) are all knobs exposed by the factory functions. That matters for production teams because the defaults are rarely right for every workload. A repo refactoring task benefits from aggressive offloading of diff files; a conversational agent might prefer smaller summaries and more in-context retention.

flowchart TD
    A[Tool returns large output] --> B{Is output > size threshold?}
    B -->|Yes| C[Write full output to /conversation_history/step_03_diff.patch]
    C --> D[Replace in context with short summary + file pointer]
    B -->|No| E[Keep output in context as-is]
    D --> F{Token count > 85% of context limit?}
    F -->|Yes| G[SummarizationMiddleware triggers compression pass]
    G --> H[Older turns synthesized into structured summary]
    H --> I[Recent N turns preserved intact]
    F -->|No| J[Continue normal execution]
    I --> K{Agent decides to compact at logical boundary?}
    K -->|Yes| L[Agent calls compact_conversation tool]
    L --> M[Compression pass runs atomically at turn boundary]
    K -->|No| N[Continue without additional compaction]
    M --> O[State channels: plan, async_subagent_jobs]
    O --> P[Unaffected by compaction - separated from message history]
    N --> O

The interaction between these two layers is what makes the whole system feel coherent rather than like two different hacks bolted together. Filesystem offloading keeps the raw data around; summarization keeps the active context lean; state separation keeps execution integrity.

The Failure Mode: Pointer Amnesia and Hallucinated File Content

Here is the trap I would warn most aggressively about. The architecture is elegant, but it creates a subtle failure mode that I've seen manifest in practice: pointer amnesia, where the model knows a file exists but does not actually read it before reasoning about its contents.

The scenario goes like this. An agent is refactoring a module. Five steps back, it ran a TypeScript build and got a compiler error. The output was large—a stack of errors spanning several hundred lines—so it was offloaded to /conversation_history/step_08_build_output.log and replaced in context with a summary: "Build failed with 14 TypeScript errors." That summary is truthful and it's in the active context. The agent continues working.

Now it needs to fix those errors. It has a summary about the errors, not the errors themselves. It knows the count but not the line numbers, the specific error codes, or the exact file paths. What happens next is the failure. The agent—operating under the pressure of a long working context, with a reasonable-but-incomplete mental model—does not reach for the file. It guesses. It reasons: "14 TypeScript errors, probably type mismatches from the refactor I just did; I'll apply a few likely fixes to the module." It invents fixes based on what it assumes the errors must be, rather than what they are.

The root cause is not laziness or a broken model. It's that we've given the model a compressed representation that feels complete. A summary that says "Build failed with 14 TypeScript errors" reads like an adequate description, and the model's attention naturally slides past the fact that it lacks the detail required to act correctly. The summary satisfies its immediate need—it knows the build failed—without triggering any flag that it needs more information to take the next step.

The mitigation is not to stop summarizing. It's to encode a hard rule into the system prompt: if the agent needs exact error signatures, line numbers, code snippets, file paths, or parameter values, it MUST read the offloaded artifact before taking any action. This is not a polite suggestion—it's an invariant. The model should treat any summary that references an offloaded file as a promise that the file contains more detail, not as the detail itself. If the task requires knowledge of what's in that file, the agent should retrieve it via a tool call before reasoning with it.

The deeper lesson here is about what summaries are for. They are for orientation—helping the model know what happened in broad strokes, so it can decide what to look at next. They are not for action. Any decision that depends on the precise content of an artifact (a fix, an edit, an evaluation of whether something is correct) must be deferred until the raw data is re-read. A disciplined prompt reinforces this discipline in the model. Without that reinforcement, the architecture's elegance becomes a liability: it gives the agent room to be imprecise, and imprecision in this context manifests as hallucinated file content and plausible-sounding but wrong patches.

The Failure Mode: Lossy Compaction Cascades

There's a second, more insidious failure mode than pointer amnesia, and it comes from the recursive nature of summarization itself. When the context window fills, the middleware compacts older turns into a summary. At a later point, that summary is compacted again into a summary-of-a-summary. Each compression pass is lossy—it foregrounds what the summarizer deemed salient and backgrounds everything else. The problem is that the passage of tokens is not linear; the constraints that matter for the final outcome are often the ones that were established earliest, and they're the ones that get smoothed away first.

The classic scenario: a user tells the agent at the start of a session, "Do not modify the authentication schema." That instruction is in the conversation, framed by the system prompt, and the agent dutifully respects it for the first ten steps. By step thirty, in the grip of a long refactoring session, the middleware compacts the early turns. The phrase "do not modify the authentication schema" becomes part of a sentence: "User provided requirements including refactoring the module structure." The specific constraint—the negative instruction that was the very point of the requirement—is gone. It gets abstracted into a positive generalization that obscures the prohibition. By step sixty, the model is comfortable rewriting the auth schema because it no longer knows the rule exists.

The root cause is not summarization being careless. It's that we're treating all information in the conversation as the same kind of information. A transient execution log and an immutable user requirement have different epistemic statuses, but a generic summarization prompt doesn't know the difference. It treats both as history to be compressed, and in compressing, it trades fidelity for brevity in ways that are mostly harmless for logs but catastrophic for constraints.

The fix is a structural one, not a prompt tweak. Separate the immutable task charter from the transient execution log at the state level. The user's original intent, the explicit constraints and invariants, the plan milestones—these live in a dedicated state channel that is excluded from the compaction pipeline entirely. They are never summarized. They're loaded into the system prompt at every step, always in full. The compaction middleware only ever touches the message list, which becomes a rolling transcript of transient execution detail.

What this buys you is a clean division of labor. The transient log becomes aggressively compressible—it's just noise that the agent can re-read from disk if needed. The charter stays fixed, unambiguous, and always in-context. You can summarize a file edit at fifty words, but you must never summarize a constraint at fifty words, because abstracting a prohibition into a general description loses the aspect of the instruction that actually governs behavior. When I think about designing a compaction pipeline, this two-tier separation is the first architectural decision I'd make: protect the charter, compress the log, and never let the two share the same fate.

Operational Gotchas: Cost, Latency, and Storage Lifecycle

The cleverness of the offloading architecture creates operational costs that can sneak up on teams. Let me walk through what I'd watch for in production, because the failure modes here are not subtle once they're underway.

Filesystem bloat is the quiet one. Every offloaded artifact—tool outputs, file diffs, test logs—persists on the sandbox workspace. Over a long-horizon task, that accumulates. I've seen scenarios where a single repository refactoring session generates tens of megabytes of raw command traces and diff files, and if the workspace uses a shared volume with quota limits, that's an operational incident waiting to happen. The fix is not optional: bind the storage workspace to the lifecycle of the task or session, and implement automatic cleanup policies for /conversation_history/ and ephemeral artifact directories upon session termination. If you're running multiple agents in parallel on the same sandbox, the risk compounds—orphaned artifacts from one session can silently consume the disk budget for everyone else.

Latency and cost spikes from compaction passes. Each compression pass requires an out-of-band LLM call to synthesize the summary. That's a block on the execution path: the agent is waiting, not working. In my experience, a compaction pass can add five to fifteen seconds of latency to a step, and the billing is doubled—one call for the summarization, one for the subsequent agent step. For interactive tools or rapid iteration loops, that's noticeable. It's not just the user experience; it's also the raw spend. If compaction happens frequently (which it will on long tasks), the cost adds up.

The prefix cache problem is the sharpest gotcha. LLM providers heavily optimize prompt prefix caching to reduce latency and cost on repeated requests. Compaction rewrites the conversation history, which invalidates that cached prefix. The next request after a compaction can cost up to four times more than the pre-compaction steady state until the cache stabilizes. This is a real cost feature, not a theoretical one—teams that run frequent compactions on long conversations can see their token spend balloon even though the total tokens per request actually went down. I'd want telemetry on this before rolling out aggressive compaction in production.

Subagent return-payload bloat. While subagents get isolated context, the spawn prompt and the return payload both pass through the parent orchestrator's context. If a subagent returns a huge final message—a long report, a big patch diff—it lands in the parent's message list and can immediately trigger a compaction pass. The architecture expects the subagent to write its outputs to disk and return only an artifact URI, but if that convention isn't enforced in the subagent's system prompt, the parent pays the token cost anyway.

The mitigation pattern is simple: design for the costs up front. Set a context threshold that leaves headroom for a compaction pass without triggering it every single step. Budget for the latency spike as a known event, not a surprise. And enforce the lifecycle cleanup so the filesystem doesn't become the second failure point after the context window.

How It Compares: Deep Agents vs. the Field

Every framework has to make a fundamental choice when the context fills up: what to drop, where to store it, and who decides. Deep Agents sits in a particular spot that's worth understanding in relation to the alternatives.

Letta (formerly MemGPT) models memory as a hierarchy of virtual memory. It distinguishes between core memory (the agent's in-context working set: persona, scratchpad) and archival memory (out-of-context storage, queryable via vector search and SQL). When the context fills, older messages get evicted into archival database tables, with only a small FIFO buffer retained in-context. The model must explicitly use tools to update memory (core_memory_append, archival_memory_search). It's a clever design that excels at long-lived conversational agents—personal assistants, chatbots, anything where the semantic content of the conversation matters more than the procedural sequence of tool calls. But it stores evicted messages as searchable facts, not as raw work history. For a debugging session where you need the exact stack trace from step twelve, vector search over "the build failed" is a weaker retrieval mechanism than a filesystem path you can cat. Letta trades procedural fidelity for cross-session semantic retrieval.

Aider takes the opposite approach: hyper-lean context, relying on repository maps built from tree-sitter AST tags. It doesn't do recursive summarization at all. When the context fills, it just prunes—dropping old conversation history entirely, relying on git and the repo map to keep the agent oriented. This is incredibly token-efficient, but it means the agent loses conversational continuity entirely. Aider assumes the repository is the source of truth, not the conversation. That works for interactive coding where the history is just a scratchpad, but it fails for long-horizon tasks where the thread of decisions matters.

OpenHands shares the filesystem philosophy with Deep Agents. It uses an event-stream architecture inside a Docker sandbox and offloads bash outputs and browser DOM dumps to disk, truncating logs that exceed preset limits. What OpenHands doesn't formalize as elegantly is the middleware layer—Deep Agents wraps this into modular, reusable SummarizationMiddleware and an explicit compact_conversation tool that the agent can call at logical boundaries. OpenHands relies more on the harness's event-stream architecture to manage state; Deep Agents makes compaction a first-class, self-regulating capability of the agent.

AutoGen and CrewAI take the declarative multi-agent approach: distribute cognitive load across specialized agents (planner, coder, reviewer), each with its own context. The orchestrator only sees the final handoff. Context isolation by design is a solid answer for modular workflows, but it means subtask details are lost unless they're manually logged by subagents. It doesn't solve the problem of a single agent running deep; it sidesteps it. Deep Agents integrates both: subagent isolation for bursty tasks and explicit intra-agent compaction middleware for long-horizon work.

Mem0 and Zep are decoupled memory engines. They extract semantic facts and entity relationships into vector/graph databases asynchronously, out-of-band. Retrieved facts get injected into the prompt only when relevant. For persistent user profiling and cross-session knowledge retrieval, this is excellent. But for continuous debugging or refactoring loops, fact extraction loses sequence ordering and tool-call causality—the exact temporal relationships that matter when you're tracing why a test failed. Deep Agents' structured message summaries plus raw disk pointers preserve that timeline.

Mechanism / System	Primary Storage Target	Compaction Trigger	History Loss / Eviction Risk	Best Suited For
Deep Agents	Sandboxed filesystem / artifact directory	Token threshold (~85%) or on-demand (compact_conversation)	Low: Raw payloads on disk; state channels survive independently	Multi-step coding, repository refactoring, deep research
Letta (MemGPT)	Vector DB + relational DB (Postgres/SQLite)	Context token threshold (triggers memory compaction loop)	Low–Medium: Evicted to searchable archival memory; conversational order can be diluted	Long-lived conversational agents, personal companions
Aider	Git repo + in-memory chat list	Manual or simple sliding-window truncation	High for chat: Erases conversational history; relies on git and repomap	Fast, interactive developer CLI coding
OpenHands	Sandboxed Docker container, event-stream disk offload	Log truncation on line/byte thresholds	Medium: Raw outputs may be truncated rather than preserved	Autonomous coding tasks in containerized environments
AutoGen / CrewAI	Ephemeral subagent state, message pass boundaries	Task delegation boundary	Medium: Subtask details lost when subagent finishes unless manually logged	Discrete, modular, multi-agent pipeline workflows
Mem0 / Zep	Vector store / knowledge graph	Continuous async extraction pipeline	Medium: Converts history to semantic facts; loses procedural temporal context	Cross-session personalization, customer support

Where Deep Agents sits in the field is as a middleweight between the hyper-lean pruners (Aider) and the heavy semantic memory engines (Letta, Mem0). It's designed for the continuous execution loop—where the agent is doing multiple sequential tool-call cycles, each building on the last. It keeps more overhead around (latency) than pure truncation, but it preserves the exact temporal and workspace state that semantic engines smooth away.

Project Playbook: Applying the Deep Agents Approach in Your Own Systems

If you're inspired to build something with this logic, here are three concrete projects I'd consider. They're not abstract exercises—each builds the machinery you'd actually need to watch these failure modes in your own environment.

A "compaction-aware" coding agent harness. The most direct application: integrate Deep Agents' summarization middleware and filesystem offloading into a custom LangGraph agent that performs repository refactoring. The design is straightforward—use a temporary workspace volume (a Docker volume or a tmpfs-backed directory) to store offloaded tool outputs, and wire SummarizationMiddleware to handle the active context. The critical piece is precisely the thing I emphasized earlier: enforce transactional boundaries before compaction runs. Never let a compaction fire mid-sequence between an AIMessage(tool_calls=...) and its paired ToolMessage. And in the system prompt, encode the read-before-act invariant for offloaded artifacts. The watch-for here is pointer amnesia, with all the downstream hallucinated patches that implies.

A compaction event monitoring dashboard. This sounds less glamorous than building the agent itself, but it's the only way you'll actually learn how your system behaves under load. The dashboard logs every compaction event: the token threshold that triggered it, the tool call that preceded it (if agent-initiated), the resulting context size after compaction, and the magnitude of the filesystem offload. Track the frequency of compact_conversation calls over time and correlate them with task phases. The pattern you're looking for is under-compaction (low frequency, high token usage—the agent running against the ceiling, degrading in attention) versus over-compaction (high frequency, tiny context—the agent thrashing between steps and erasing its own short-term memory). Set an alert for when compaction happens more than a handful of times within a short window: that's a thrashing loop you want to know about immediately. Integrate with Deep Agents' middleware hooks to capture the events; store metrics in a time-series database; visualize with a simple frontend or even a Grafana panel.

A hybrid trigger system with dual thresholds. Implement a custom middleware that combines an automatic hard threshold (say, 80% of context usage) with agent-initiated compact_conversation at logical phase transitions. The key is protecting the immutable task charter: use LangGraph state channels so that the user's original goals and confirmed constraints are never compacted—they get loaded into the system prompt at every step in full. Test it on a multi-stage research-then-implementation task. The watch-fors are exactly the gotchas I listed earlier: verify automatic triggers respect turn boundaries, confirm the state channels survive compaction, and measure the cost impact. Track the token spend before and after compaction to catch the prefix cache busting costs when they happen. If the dual trigger works well, you'll see an agent that knows when to compact itself—after finishing a research phase, before starting implementation—rather than having the harness do it at an arbitrary token count.

Each of these projects is a way to pressure-test the philosophy in your own stack. They compound: you can't know whether your harness is prone to under- or over-compaction without the monitoring dashboard, and you can't implement the dual throttle without understanding the middleware's behavior in practice. If you're starting with just one, the monitoring dashboard is the highest-leverage build—it'll tell you whether you need the other two at all.

Resources

• GitHub - langchain-ai/deepagents: The batteries-included agent harness. · GitHub