Memory compression

Token budgets are finite. Conversations grow. After enough turns, the agent’s context is full of itself — old tool results, old reasoning, old file contents that have since changed. Most of it is noise. Compression is the act of throwing away the noise without throwing away the signal.

flowchart LR
T1[Turn 1] --> T2[Turn 2] --> T3[Turn 3] --> T4[Turn N-2] --> T5[Turn N-1] --> T6[Turn N]
T1 -. compress .-> S[(Summary)]
T2 -. compress .-> S
T3 -. compress .-> S
S --> Ctx[Context for next call]
T4 --> Ctx
T5 --> Ctx
T6 --> Ctx
class S sum

The four strategies

def compress(turns, target_tokens):
    if total_tokens(turns) < target_tokens:
        return turns
    keep_recent = turns[-RECENT_N:]
    to_summarize = turns[:-RECENT_N]
    summary = aux_llm.complete(
        SUMMARIZE_PROMPT.format(content=to_summarize),
        # the prompt enumerates what must survive
    )
    return [SystemMessage(summary), *keep_recent]

flowchart LR
EL[(Event Log<br/>append-only)] --> CV1[View · full]
EL --> CV2[View · summarized]
EL --> CV3[View · type-filtered]
CV2 --> LLM

if (tokenCount(messages) > BUDGET) {
  messages = messages.slice(-RECENT_N);
}

const RECENT = 10;
const recent = messages.slice(-RECENT);
const old = messages.slice(0, -RECENT);
const summary = await llmSummarize(old, { preserve: ['ids', 'paths'] });
return [systemMsg, summary, ...recent];

The preservation list — your domain model

The summarizer prompt has two halves. The first half says “compress aggressively.” The second half says “but never lose these specific things.” The second half is what separates a useful summary from useless mush.

You are summarizing an agent's session for memory compression.

PRESERVE EXPLICITLY (do not paraphrase, do not drop):
- Task tracker IDs and their statuses
- File paths the agent has read or written
- Exact text of error messages (byte-for-byte)
- Tool arguments used (not just tool names)
- In-progress actions and their identifiers

COMPRESS LOOSELY:
- The agent's narration
- Repeated tool failures
- Casual reasoning

What goes in the first list is the agent’s domain knowledge. Across the corpus:

When does compression fire?

1. Token threshold

   The most common: when input token count exceeds a fraction (e.g. 75%) of the model’s context limit, compress. Easy to compute, predictable. Beware: each compaction busts your prompt cache.

2. Turn count

   “Every 20 turns” or “every 50 turns.” Simpler than tokens but ignores variable turn size — one tool result might be 50K tokens by itself.

3. Cost ceiling

   Track per-session spend; compress when it crosses a configured cap. Pragmatic for SaaS — converts a UX problem into a billing one.

Pick a strategy

Cross-project comparison

Projects that implement this

• Claude Code — Anthropic's official agentic CLI. Streaming tool calls, prompt caching, thinking signatures, multi-agent subagents, slash commands.
• OpenHands (v0) — All-hands AI v0 — autonomous software engineer agent. Event-sourced state, microagents, controller-level guardrails.
• Strix — Open-source 'AI hacker' for autonomous pentesting. XML tool format, markdown-as-skills, LLM-based dedupe, module-level agent graph.
• OpenHands (v1) — OpenHands re-architected: cleaner controller, refined memory condenser, improved tool dispatch. v1 of the All-Hands agent.
• Hermes Agent — 40+ tool, multi-platform agent. Provider adapters per LLM, trajectory compression preserves first/last turns, side-channel auxiliary client.

Related insights