Context Engineering vs Prompt Engineering: What Changed

Two years ago the highest-leverage skill in building with LLMs was writing the prompt. It still matters. But on the systems people are building now (agents that call tools, read documents, and run for many turns), the prompt is a small fraction of what the model sees on a given turn, and optimizing it alone is like tuning the engine of a car with four flat tires. The skill that replaced it has a name: context engineering.

What Changed

On a single-turn chat feature, the context is the system prompt plus the user's message. Prompt engineering covers most of it.

On an agent nine turns into a task, the context looks like this:

system prompt            1,800 tokens
tool definitions         6,400 tokens   (22 tools)
retrieved documents      9,000 tokens
user memory                600 tokens
conversation so far     14,000 tokens
latest tool result       3,200 tokens

The system prompt is 5% of the context. The rest is tool definitions the task mostly does not need, retrieved chunks of mixed relevance, history nobody is controlling the growth of, and raw tool output. Every one of those is a design decision, and prompt engineering has nothing to say about any of them.

What Context Engineering Optimizes

Prompt engineering asked	Context engineering asks
Is the instruction clear?	Is the instruction the right fraction of the budget?
Are the examples good?	Which tools does this task need to see?
Is the format specified?	Is the retrieved set precise, or just large?
Does it handle edge cases?	What survives compaction, and what is retrievable after?
Did the output improve?	Did cost per successful run improve, and did cache hit rate hold?

The new optimization targets are budget allocation, tool surface, retrieval precision, memory policy, compaction strategy, and cache-friendly layout. The new metrics are tokens per run by section, cache hit rate, compaction events per run, and cost per successful run.

Before and After

A document-processing agent with 22 tools, top-20 retrieval, no compaction policy, and the system prompt rebuilt per request with the user's name and a timestamp at the top.

Before: 41,000 tokens per turn average, 0% cache hits (the timestamp busted the prefix), 5 compactions per run, task success 79%, cost per successful run $0.61.

Changes, in order:

1. Scope tools per task type: the extraction task sees 6 tools, not 22. (Tool definitions: 6,400 to 1,900 tokens.)
2. Retrieval as recall then precision: retrieve 50, re-rank to 5, pack with source IDs. (Documents: 9,000 to 2,800 tokens, and accuracy went up.)
3. Reduce tool results in code before they enter the context: return summaries with IDs, fetch detail on demand. (Latest tool result: 3,200 to 700 tokens.)
4. Move the user's name and timestamp out of the system prompt and into the final user turn. Stable content first. (Cache hits: 0% to 88%.)
5. Compaction keeps raw artifacts retrievable by ID and carries constraints verbatim.

After: 17,000 tokens per turn, 88% cache hits, 1 compaction per run, task success 84%, cost per successful run $0.19.

Nothing in the system prompt's wording changed. That is the point.

The Principles

The context is a budget. Every token competes. Write down where they go before optimizing anything.

Scope the tool surface per task. Fewer tools means fewer tokens and fewer wrong picks. Tool search and programmatic tool calling keep large inventories out of the context until needed.

Retrieve wide, pack narrow. Recall with a cheap retriever, precision with a re-ranker, pack a small set with provenance.

Reduce before you include. Tool output goes through code that keeps what the model needs.

Memory has write policies. The model proposes long-term memory; the harness commits. Otherwise one bad run poisons all future runs.

Compact without amnesia. Raw artifacts stay retrievable; constraints survive verbatim.

Layout for the cache. Stable to variable, nothing per-request in the prefix. Cache hit rate is an operational metric.

Is Prompt Engineering Dead?

No. The instruction still has to be clear, the format still has to be specified, and a bad system prompt still produces bad output. It has been demoted from the whole job to one layer of it. On simple features it is still most of the work. On agents it is the part you finish first and touch least.

What to Practice Next

Instrument one agent: tokens per section per turn, cache hit rate, compaction events. Then apply the five changes above in order and measure cost per successful run on a fixed task set. The full method, with the memory and compaction details, is in context-engineering-designing-what-the-model-sees.