Context Engineering for AI Agents: Compaction, Notes, Sub-Agents (2026)

October 15, 2026 Khimananda Oli 12 min read Database, Virtualization

By Khimananda Oli | Last reviewed: October 2026

Context engineering is curating what an agent actually sees at each step, as distinct from prompt engineering, which is writing the instructions. A long-running agent degrades well before its context window is full — every token it accumulates competes for a finite "attention budget," and stale tool results crowd out the information that matters. The fix is three complementary techniques: compaction (summarise and restart), structured note-taking (persist state outside the context window), and sub-agent isolation (delegate exploration to a clean context that returns only a condensed result).

Why does an agent get worse long before its context window fills up?

The intuitive model of a context window is a bucket: it holds a fixed number of tokens, and the agent works fine until the bucket is full. That model is wrong in a way that matters. Anthropic's own engineering guidance frames it differently — every token in context draws on a finite "attention budget," and because transformer self-attention means each token attends to every other token, adding more tokens creates an n² growth in pairwise relationships the model has to weigh. Context, in their words, "must be treated as a finite resource with diminishing marginal returns." A context window at 40% capacity, mostly filled with stale tool output from three steps ago, can already be hurting the agent more than a tight, relevant 10% would.

This has an independent line of research behind it, not just a vendor's framing. Chroma Research tested 18 models — including Claude Opus 4, Sonnet 4, GPT-4.1, Gemini 2.5 Pro and Qwen3 — on tasks where they held the actual difficulty constant and varied only the amount of surrounding content. The finding: performance consistently degraded as input length grew, even when the needle being searched for never changed and never moved. On their LongMemEval benchmark, models did significantly better on a focused ~300-token prompt than on a ~113,000-token prompt containing the same answer — and Claude models in particular showed the largest gap, largely because the model would abstain rather than commit to an answer buried in irrelevant surrounding text. The practitioner term for this is context rot: not an overflow, a degradation in the signal-to-noise ratio of what the model is reasoning over.

If you have not yet covered how tokens, context windows and long-context model limits work at a basic level, tokens, embeddings and context windows explained and long-context LLMs: strategies and limits cover that ground. This article is specifically about the harder problem underneath it: an agent that runs for dozens of steps accumulates a context nobody wrote by hand, and keeping that accumulated context clean is a discipline of its own.

Context rot in a long-running agent: the context window is far from full at step 23, but the useful signal has been diluted by accumulated stale tool output.

What is compaction, and when should it run?

Compaction is the most direct response: summarise the context's contents, then reinitiate a new, smaller context window built from that summary instead of the full history. Anthropic describes Claude Code's own approach as keeping key architectural decisions and unresolved issues while discarding redundant tool outputs — concretely, retaining only the five most recently accessed files in full, and summarising everything before that.

Clearing stale tool results is the lightest form of compaction and the one to reach for first: a tool call from fifteen steps ago whose output has already been acted on rarely needs to stay in context verbatim. The heavier form — a full summarisation pass — trades a pause (compaction is usually synchronous: the agent stops, a compactor model generates the summary, then the agent resumes) for a meaningfully cleaner context. There is no single correct token threshold at which to trigger it; Anthropic's own guidance ties it to proximity to the model's limit rather than a fixed number, and recent research on long-horizon coding agents argues that purely reactive triggers — waiting until context is already saturated — and purely periodic triggers — which can interrupt an agent mid-subgoal — both have real failure modes. A reasonable default is to compact when stale tool output rather than new reasoning starts to dominate recent turns, not at a fixed token count.

def should_compact(history, window_limit, headroom_pct=0.25):
    """Conservative trigger: compact once stale tool output crosses a
    share of the remaining budget, not at the hard window limit."""
    used = sum(tok_count(turn) for turn in history)
    stale_tool_tokens = sum(
        tok_count(turn) for turn in history
        if turn.role == "tool" and turn.superseded
    )
    remaining = window_limit - used
    return (
        remaining < window_limit * headroom_pct
        or stale_tool_tokens > used * 0.4
    )

def compact(history, compactor_model):
    keep = [t for t in history if t.pinned or t.recent_file_edit]
    to_summarize = [t for t in history if t not in keep]
    summary = compactor_model.summarize(
        to_summarize,
        instructions="Preserve: open issues, key decisions, unresolved "
                     "errors. Drop: resolved tool output, dead-end "
                     "exploration, superseded file reads.",
    )
    return [Turn(role="system", content=summary)] + keep

What does structured note-taking add that compaction doesn't?

Compaction manages what stays inside the context window. Structured note-taking moves state outside it entirely — the agent writes to a persistent store (a to-do list, a NOTES.md file, a scratchpad) and reads back only what the current step needs, rather than ever holding the full history of the task's progress in context. Anthropic's example of this in production is an agent playing a long-running Pokémon game, which tracked a running record of its progress across more than a thousand steps — including things like which Pokémon had reached which level toward a target — entirely in an external file, pulling in only the relevant lines when deciding its next move.

The practical advantage over compaction is that nothing gets summarised, and therefore nothing gets lossily compressed. A compaction pass can plausibly drop a detail that turns out to matter forty steps later; a note written to disk is either read back verbatim or it isn't read back at all. The cost is that you have to decide, as the system designer, what's worth writing down and when — compaction is comparatively "free" in that it needs no per-task design, while note-taking needs a schema for what the agent should persist.

# Minimal structured note-taking for a long-running agent task
import json, pathlib

NOTES_PATH = pathlib.Path("agent_state/task_notes.md")

def write_note(section: str, content: str):
    """Append a dated entry under a named section, e.g. 'decisions',
    'open_issues', 'completed'. Read back selectively, not wholesale."""
    NOTES_PATH.parent.mkdir(exist_ok=True)
    with NOTES_PATH.open("a") as f:
        f.write(f"\n## {section}\n{content}\n")

def read_notes(section: str | None = None) -> str:
    if not NOTES_PATH.exists():
        return ""
    text = NOTES_PATH.read_text()
    if section is None:
        return text
    # pull back only the requested section, not the whole file
    parts = text.split(f"## {section}")
    return parts[1].split("## ")[0] if len(parts) > 1 else ""

How does sub-agent isolation solve a different problem again?

Compaction and note-taking both manage one agent's own accumulating context. Sub-agent isolation sidesteps the accumulation at the source: a lead agent delegates an exploratory or research-heavy sub-task to a sub-agent that runs in its own clean context window, does whatever digging it needs — tens of thousands of tokens of exploration, per Anthropic's own description — and returns only a condensed summary, typically 1,000 to 2,000 tokens, back to the lead agent. The lead agent's own context never sees the sub-agent's intermediate exploration at all.

This is the right tool specifically when a task has a bounded, answerable sub-question that doesn't need to influence the lead agent's ongoing reasoning step by step — "find every file that imports this deprecated function" is a good fit; "decide whether to approve this refund given everything we've discussed" is not, because splitting that into an isolated sub-agent call would just throw away the context it needs. The architectural trade-offs of running several agents together, including coordination failure modes that have nothing to do with context at all, are covered separately in multi-agent systems: patterns and pitfalls.

Compaction, structured note-taking and sub-agent isolation solve different parts of the context engineering problem and are typically combined rather than chosen one at a time.

Choosing a context engineering strategy by task shape: duration first, then whether the work splits into a bounded sub-question a sub-agent can own.

How do you choose between these for a given agent?

Match the strategy to the shape of the work, not to a single default. A concrete case makes the decision tree above less abstract: a coding agent asked to "migrate this codebase off a deprecated ORM" is a long-running task with a clear bounded sub-question inside it — finding every call site. That sub-question goes to a sub-agent, which can grep and read through hundreds of files using tens of thousands of tokens nobody but it ever sees, returning a condensed list of call sites and the pattern each one follows. The lead agent, meanwhile, keeps a running note of which files have been migrated and which conversion edge cases it has already handled, read back selectively rather than re-included wholesale on every turn. Compaction runs underneath both, clearing the file-read history that's no longer needed once a file is migrated and confirmed. None of the three techniques alone would keep this task's context usable past the first dozen files; together they are what lets the same architecture keep working at file three hundred as it did at file three.

Match the strategy to the shape of the work, not to a single default:

Short, bounded tasks that finish in a handful of steps rarely need any of this — context engineering is a long-horizon-agent problem, and adding compaction to a three-step agent is pure overhead.
Long-running single-threaded agents — a coding agent working through a large refactor, a research agent following a chain of sources — are the core case for compaction and structured notes, usually both: notes for the durable record of decisions and open issues, compaction as the mechanism that keeps the live context from drowning in resolved tool output.
Tasks with a clear, bounded sub-question that doesn't need to shape the main agent's ongoing reasoning are the case for a sub-agent — "search the codebase for every caller of this function" should not cost the lead agent's context a single token of exploration, only the answer.
Tasks requiring continuous judgment across the whole history — anything where an early decision constrains a later one in a way that's hard to summarise — are the case where you should be cautious about compaction specifically, because that's exactly the kind of detail a summarisation pass is most likely to drop.

Whichever combination you pick, instrument it. If you are already tracking LLM calls with OpenTelemetry's GenAI semantic conventions, tag compaction events and sub-agent invocations as their own span types so you can see, after the fact, how often each strategy fires and whether task completion dips around the points where compaction ran — that correlation is the practical signal for whether your trigger is too aggressive or too late. The same evaluation discipline from evaluating AI agents: trajectory, tool calls, task completion applies directly here: a regression in task completion that appears specifically on long-running test cases, but not short ones, is the signature of a context engineering problem rather than a model or tool problem.

Context engineering also interacts directly with cost, since every token retained is a token billed again on the next call in a non-cached conversation — the trigger discipline above is as much a cost control as a quality one, and pairs naturally with the caching strategies in prompt caching to cut LLM costs and the broader picture in LLM cost optimization for production apps. If you are just getting started with agents and haven't yet built a tool-calling loop, building your first AI agent with tool use and agent memory: short-term vs long-term are the right prerequisites — context engineering is the discipline you need once that loop runs long enough for the problem described here to actually show up.

If you want context engineering designed into an agent from the start — compaction triggers, a notes schema, and sub-agent boundaries chosen for your actual workload rather than copied from a demo — my DevOps and cloud consulting services cover exactly this.

Frequently Asked Questions

The set of strategies for curating and maintaining the optimal set of tokens an agent sees at each step during a long-running task, as distinct from prompt engineering, which is about how instructions are written. It covers everything in context, not just the system prompt.

Because every token competes for a finite attention budget, and transformer self-attention means more tokens create quadratically more pairwise relationships to weigh. A window that is far from full but mostly filled with stale tool output can already be hurting the agent more than a smaller, cleaner one would.

The practitioner term for an agent's output quality declining as irrelevant tokens accumulate in its context, lowering the signal-to-noise ratio. Chroma Research measured this directly across 18 models, holding task difficulty constant and varying only input length, and found consistent degradation.

Summarising an agent's accumulated context and reinitiating a smaller context window built from that summary. Claude Code's implementation keeps key decisions, open issues and the five most recently accessed files, discarding redundant resolved tool output.

There is no universal token threshold — Anthropic ties it to proximity to the model's context limit rather than a fixed number. A practical trigger is when stale, already-acted-on tool output starts to dominate recent context rather than new reasoning, not a fixed token count.

Having the agent persist state — decisions, progress, open issues — to an external file or store outside its context window, reading back only what's relevant at each step rather than carrying the full history in context. Anthropic's example is an agent tracking game progress across more than a thousand steps in an external file.

Compaction compresses what's inside the context window, which risks lossily dropping a detail. Note-taking moves state outside the window entirely, so nothing is summarised — a note is either read back verbatim or not read at all, at the cost of needing a schema for what to persist.

Delegating a bounded exploratory task to a sub-agent with its own clean context window. The sub-agent can use tens of thousands of tokens exploring, but returns only a condensed summary — typically 1,000 to 2,000 tokens — so the lead agent's context never holds the exploration itself.

When a task has a bounded, answerable sub-question that doesn't need to shape the lead agent's ongoing reasoning step by step, such as searching a codebase for every caller of a function. Tasks requiring continuous judgment across the whole history are a poor fit for sub-agent isolation.

No. A longer context window raises the ceiling before context engineering becomes necessary, but research shows performance degrades with input length even well under that ceiling, on models with large advertised windows. Context engineering manages quality within whatever window you have.

It's a long-horizon-agent problem — a task finishing in a handful of steps rarely needs it. Watch for task completion scores dropping specifically on your longest-running test cases while short ones stay fine; that pattern points at context rot rather than a model or tool issue.

Yes, and most production agents do. A common pattern is sub-agents for bounded research, structured notes for the durable record of task state, and compaction as the backstop for whatever accumulates in the main agent's loop regardless.

Compaction is usually synchronous: the agent pauses, a compactor model generates a summary of the context to be dropped, and the agent resumes with the smaller, summarised context. This is a real latency cost, which is part of why trigger timing matters.

Eighteen LLMs including Claude Opus 4, Sonnet 4, GPT-4.1, Gemini 2.5 Pro and Qwen3, on tasks that held the target information and task difficulty fixed while varying only the surrounding input length, isolating length itself as the cause of the performance decline they measured.

Yes. Every token retained in context is billed again on each subsequent call in an uncached conversation, so compaction and note-taking that keep context lean also reduce cost, and the same trigger decisions interact directly with prompt caching strategy.