Context and Harness Engineering

In the agents module you built the loop: the model proposes a tool call, your code runs it, the result goes back, repeat. That loop is the model's contribution. This module is about yours. Two disciplines have emerged around the loop, and they are what separate agents that ship from agents that demo.

Context engineering is designing everything the model sees on a turn: the system prompt, the tool definitions, retrieved documents, memory, prior turns, and tool results, all under a token budget. Harness engineering is building the deterministic runtime around the model that decides which tools exist, validates every call, enforces budgets, pauses for approval, verifies results, and records everything. The formula people use is agent = model + harness, and this module builds both halves.

Part 1: Context Engineering

The context is a budget

Write down where the tokens go on a typical turn. On a real agent nine turns in, the system prompt is often the smallest item and tool definitions plus accumulated tool results are the largest. Two consequences follow: most of the tool definitions are irrelevant to the current task, and nothing is controlling the growth of the conversation. Both are decisions you have not made yet.

Scope tools per task

Loading every tool into every session costs tokens and accuracy: a model choosing among forty tools picks wrong more often than one choosing among six. Scope in code: a task type maps to an allowlist, and the model only sees those definitions. This is also a security boundary, because tools the model cannot see are tools an injection cannot invoke.

Reduce tool results before they enter the context

A tool that returns 4,000 rows should return them to code that reduces them to the twenty the model needs. Return summaries with IDs; offer a way to fetch detail on demand.

Memory is three things

Store	What	Written by
Working	This run's context	The loop
Episodic	Records of past runs	The harness, at run end
Long-term	Curated facts and preferences	Deliberately, with review

The model may propose long-term memory writes. The harness or a human commits them. Free writes are how one bad run poisons every future one.

Compaction is lossy

When the window fills, earlier turns get summarized. Keep raw tool results retrievable by ID so the summary can point at them, and carry constraints forward verbatim ("output must be CSV; do not touch data/raw"). Count compaction events per run; many compactions means the context strategy is wrong.

Order for caching

Providers cache the prefix of a prompt and bill cached tokens at a discount. Put stable content first (system prompt, tools, reference docs), variable content last, and nothing per-request in the stable part. Monitor cache hit rate.

Part 2: Harness Engineering

The five layers

Layer	Owns
Tool orchestration	Registry, schemas, per-task allowlists, execution
Guardrails	Permissions, turn and dollar budgets, schema validation, approval gates
Verification	Checks between steps: parses, runs, matches, judged
Context and memory	Part 1, enforced in code
Observability	Traces of proposed, refused, paused, and executed calls; costs

A minimal harness

pythonCopy
class Harness:
    def __init__(self, model, tools, max_turns=25, max_usd=2.0):
        self.model, self.tools = model, tools
        self.max_turns, self.max_usd = max_turns, max_usd

    def run(self, task, allowed, state):
        messages = [{"role": "user", "content": task}]
        visible = [self.tools[n].schema for n in allowed]          # scoped surface
        while True:
            if state.turns >= self.max_turns or state.cost >= self.max_usd:
                state.log("budget_exceeded"); raise BudgetExceeded(state.run_id)
            reply = self.model.complete(messages, tools=visible)
            state.turns += 1; state.cost += reply.cost
            if not reply.tool_calls:
                state.log("final", text=reply.text); return reply.text
            for call in reply.tool_calls:
                state.log("proposed", tool=call.name, args=call.args)
                if call.name not in allowed:
                    result = state.refuse(call, "not available to this task")
                elif (errs := validate(call.args, self.tools[call.name].schema)):
                    result = state.refuse(call, f"invalid args: {errs}")
                elif self.tools[call.name].side_effect and call.name not in state.approvals:
                    state.log("paused_for_approval", tool=call.name, args=call.args)
                    state.save(); raise NeedsApproval(state.run_id, call)     # resume later
                else:
                    result = reduce_for_context(self.tools[call.name].fn(**call.args))
                    state.log("executed", tool=call.name)
                messages += [reply.as_message(), tool_message(call.id, result)]
                state.save()

Read what each line buys. allowed scopes tools (context and security). validate refuses bad calls before they reach code. side_effect plus approvals turns irreversible actions into a pause the run resumes from after a human says yes, possibly hours later in a different process, because state.save() persists everything after every step. The budget check makes runaway loops impossible. reduce_for_context keeps tool output from flooding the window. And the log records proposed and refused calls, which is where the interesting bugs live.

Verification loops

The model produces; it does not know if it is right. Insert checks it cannot skip: the output parses, the generated code runs, the SQL returns a plausible row count, a second source agrees, or a separate judge model grades it against a rubric you validated with humans. On failure, feed the specific failure back ("row count 0, expected about 1,200"), not "try again".

The never-again loop

When the agent fails: reproduce from the trace; ask which layer should have made the failure impossible; fix that layer; add the case to your eval suite. Teams that do this converge on reliable agents. Teams that edit prompts converge on a long prompt.

Sandboxing

Anything that runs model-generated code or commands runs in an isolated container: no credentials, allowlisted network, read-only mounts, hard timeout. Output comes back as data.

Common Mistakes and Bad Instincts

• All tools, always. Tokens and wrong picks.
• Raw tool output into context. Reduce first.
• Model-written long-term memory. Permanent damage from one bad run.
• Budgets in the prompt. A request is not a limit.
• Approval with input(). A pause is a saved state, not a blocking call.
• Logging only executed calls. Refused and paused are the signal.
• Prompt first when something breaks. Layer first.

Where to Go Next

• mcp-building-a-tool-server: the standard way to expose tools to this harness
• agent-evals-and-ai-security: the eval suite the never-again loop feeds, and the harness under attack
• observability-monitoring-drift-and-llm-evals: production signals for agents

What to Practice Next

Extend the agent from the previous module with the harness above and persist state to SQLite. Instrument tokens per section and cache hit rate. Demonstrate with traces: a refused out-of-scope call, a refused invalid-args call, an approval pause resumed in a new process, a budget stop, and a verification failure corrected in the next turn. Then scope tools per task and reduce tool results in code, and report the token reduction on a fixed 20-task set. Target: 40% fewer tokens, same task success.