Agents, Tool Use, and Workflow Orchestration

An AI agent is a system where an LLM iteratively decides what to do next - calling tools, processing results, and continuing until a goal is achieved. In 2025, "agent" is one of the most over-used words in AI product work. This module distinguishes what agents are genuinely good at, where they fail predictably, and how to build constrained, reliable agentic workflows rather than hoping for emergent behavior.

What an Agent Actually Is

A minimal agent is three components in a loop:

1. LLM: decides what to do next based on current state and available tools.
2. Tools: functions the LLM can call to interact with the world (search, code execution, API calls, database queries).
3. Loop: after each tool call, the LLM sees the result and decides whether to call another tool or return a final answer.

pythonCopy
import anthropic
import json
from typing import Any

client = anthropic.Anthropic()

# Define tools as JSON schemas
tools = [
    {
        'name': 'search_documents',
        'description': 'Search the knowledge base for relevant information.',
        'input_schema': {
            'type': 'object',
            'properties': {
                'query': {'type': 'string', 'description': 'Search query'},
                'max_results': {'type': 'integer', 'default': 5},
            },
            'required': ['query']
        }
    },
    {
        'name': 'run_python',
        'description': 'Execute Python code and return stdout.',
        'input_schema': {
            'type': 'object',
            'properties': {
                'code': {'type': 'string', 'description': 'Python code to execute'},
            },
            'required': ['code']
        }
    }
]

def execute_tool(tool_name: str, tool_input: dict) -> Any:
    """Dispatch tool calls to actual implementations."""
    if tool_name == 'search_documents':
        return search_kb(tool_input['query'], tool_input.get('max_results', 5))
    elif tool_name == 'run_python':
        return run_sandboxed_python(tool_input['code'])
    else:
        return {'error': f'Unknown tool: {tool_name}'}

The Model Is Not the Agent

Here is the mental model that separates people who ship agents from people who demo them: an agent is a model plus a harness. The model proposes; the harness disposes. The harness is ordinary code you write. It owns the list of tools the model can see, checks every proposed call against a schema and a permission policy, executes the call (or refuses it), records the result, and decides what the model sees next.

Why insist on this? Because every question a real team asks about an agent is a harness question:

• "What is the worst thing it can do?" That is the tool list and the permission policy.
• "How much can one run cost?" That is a budget check in the harness.
• "What did it do yesterday at 3pm?" That is the trace log.
• "Why did it loop for ten minutes?" That is a missing verification step and a missing turn limit.

When an agent misbehaves, your first move is not to rewrite the prompt. It is to find the harness layer that should have made the misbehavior impossible and fix it there. A prompt is a suggestion; a permission check is a guarantee. You will see this framing again in the dedicated modules on context engineering, harness engineering, and MCP later in this phase.

The Agent Loop

pythonCopy
def run_agent(user_request: str, max_steps: int = 10) -> str:
    messages = [{'role': 'user', 'content': user_request}]

    for step in range(max_steps):
        response = client.messages.create(
            model='claude-opus-4-7',
            max_tokens=2048,
            tools=tools,
            messages=messages,
        )

        # If the model wants to call a tool
        if response.stop_reason == 'tool_use':
            tool_calls = [b for b in response.content if b.type == 'tool_use']

            # Add assistant message with tool calls
            messages.append({'role': 'assistant', 'content': response.content})

            # Execute each tool call and add results
            tool_results = []
            for tc in tool_calls:
                result = execute_tool(tc.name, tc.input)
                tool_results.append({
                    'type': 'tool_result',
                    'tool_use_id': tc.id,
                    'content': json.dumps(result),
                })
                print(f"Step {step+1}: called {tc.name}({tc.input})")

            messages.append({'role': 'user', 'content': tool_results})

        # If the model is done
        elif response.stop_reason == 'end_turn':
            final_text = next(b.text for b in response.content if b.type == 'text')
            return final_text

    return "Agent reached maximum steps without completing the task."

The key safety property: max_steps prevents infinite loops. Without it, a confused model can call tools indefinitely.

When Agents Are Useful

Agents are well-suited for tasks that:

• Require sequential tool use: "search for X, then compute Y from the results, then write a summary" - each step depends on the previous.
• Have uncertain paths: you don't know in advance which tools will be needed.
• Are too long for a single context: the task requires more information than fits in one prompt.

Examples: research assistant (search + synthesize), code debugging loop (run code + read error + fix + run again), data analysis pipeline (query DB + run stats + generate report).

When Agents Fail and When to Avoid Them

Agents fail predictably in these situations:

Too many tools: an agent with 20 tools will frequently pick the wrong one or enter loops. Keep tool sets small (< 8) and tools focused.

Ambiguous goals: if the success condition is unclear, the agent loops or halts prematurely. Define done criteria explicitly in the system prompt.

No recovery from tool errors: if a tool returns an error and the agent has no fallback, it either halts or generates a plausible-sounding but incorrect answer. Design tools to return structured errors the LLM can act on.

Tasks with exact deterministic requirements: if the task requires exact code output, a specific file format, or transactional correctness, an agent is the wrong architecture. Use a scripted workflow with LLM-assisted steps instead.

Workflow Orchestration: Structured Graphs vs. Full Autonomy

Full autonomy (let the LLM decide everything) maximizes flexibility but minimizes reliability. For production, the right trade-off is usually a structured workflow with LLM-powered steps rather than a fully autonomous agent.

pythonCopy
from dataclasses import dataclass
from typing import Callable

@dataclass
class WorkflowStep:
    name: str
    fn: Callable
    input_key: str
    output_key: str

def run_workflow(steps: list[WorkflowStep], initial_state: dict) -> dict:
    """Execute a deterministic sequence of steps, some powered by LLM."""
    state = initial_state.copy()
    for step in steps:
        print(f"Running step: {step.name}")
        state[step.output_key] = step.fn(state[step.input_key])
    return state

# Example: customer support ticket classification pipeline
pipeline = [
    WorkflowStep('classify', classify_ticket_with_llm, 'raw_text', 'category'),
    WorkflowStep('extract', extract_entities_with_llm, 'raw_text', 'entities'),
    WorkflowStep('route', rule_based_routing, 'category', 'queue'),
    WorkflowStep('draft', draft_response_with_llm, 'entities', 'draft_response'),
]
result = run_workflow(pipeline, {'raw_text': ticket_text})

Each step has a deterministic interface (defined input and output keys). The LLM-powered steps are constrained to specific subtasks. This gives you:

• Deterministic execution order and structure
• Ability to test each step independently
• Straightforward retry and error handling at each step

Memory Patterns

Agents need memory to maintain context across multiple turns or across sessions.

In-context memory: simply include prior tool results and observations in the message history. Works for single sessions, limited by context window size.

External key-value memory: store facts in a database; retrieve relevant ones before each LLM call.

pythonCopy
import json
from pathlib import Path

class AgentMemory:
    def __init__(self, path: str):
        self.path = Path(path)
        self.data = json.loads(self.path.read_text()) if self.path.exists() else {}

    def set(self, key: str, value: Any):
        self.data[key] = value
        self.path.write_text(json.dumps(self.data, indent=2))

    def get(self, key: str, default=None):
        return self.data.get(key, default)

    def search(self, query: str) -> list[tuple[str, Any]]:
        # Simple substring match - replace with semantic search for larger stores
        return [(k, v) for k, v in self.data.items() if query.lower() in str(v).lower()]

For production agents: use a vector store for semantic memory retrieval rather than keyword search.

Durable Execution: Surviving Crashes and Waiting for Humans

The memory patterns above cover what the model remembers. There is a separate problem: what the program remembers. An agent that runs for minutes, calls many tools, and sometimes waits for a human approval must survive the process being killed halfway through.

The fix is to checkpoint state after every step and make resuming the default. Concretely: store the run's state (which step you are on, every tool result so far, the pending question for the human) in a database keyed by a run ID. When the process restarts, load the state and continue from the last completed step instead of starting over. Frameworks like LangGraph give you a checkpointer that does this per graph node; workflow engines like Temporal do it per activity with retries built in.

The same machinery gives you human-in-the-loop for free. "Wait for approval" becomes a state the run sits in, possibly for hours, until a separate process (a web form, a Slack button) writes the approval and resumes it. That is how you make an agent safe to run unattended: it never performs an irreversible action without a recorded approval, and the recording is part of the run's history.

Two practical rules:

1. Every tool result gets an ID and is stored raw. Summaries in the context window may point to it, but the raw artifact never disappears.
2. Any tool with an irreversible side effect (send, pay, delete, deploy) requires an explicit approval state before execution. If you cannot show the approval in the trace, the action should not have happened.

Guardrails: Keeping Agents from Causing Harm

Every production agent needs safety constraints. The three categories:

Output validation: before executing a tool call, validate that the parameters are safe and within expected ranges.

pythonCopy
def validate_tool_call(tool_name: str, tool_input: dict) -> str | None:
    """Return an error message if the call is unsafe, else None."""
    if tool_name == 'run_python':
        code = tool_input.get('code', '')
        forbidden = ['import os', 'import subprocess', 'open(', 'eval(', 'exec(']
        for pattern in forbidden:
            if pattern in code:
                return f"Blocked: code contains forbidden pattern '{pattern}'"
    if tool_name == 'search_documents':
        if len(tool_input.get('query', '')) > 500:
            return "Blocked: query too long"
    return None

Rate limiting: prevent runaway tool use.

Human-in-the-loop for high-stakes actions: for irreversible actions (sending emails, modifying databases, spending money), require human confirmation before execution.

pythonCopy
def run_agent_with_approval(request: str, high_stakes_tools: set[str]) -> str:
    # ... same loop as above, but:
    for tc in tool_calls:
        if tc.name in high_stakes_tools:
            approval = input(f"Approve {tc.name}({tc.input})? [y/n]: ")
            if approval.lower() != 'y':
                # Inject a "user declined" result instead of executing
                tool_results.append({'type': 'tool_result', 'tool_use_id': tc.id,
                                      'content': 'User declined this action.'})
                continue
        result = execute_tool(tc.name, tc.input)
        # ...

Tracing and Observability

Agents are hard to debug without traces. Log every tool call, its inputs, and its outputs:

pythonCopy
import logging
import time

logger = logging.getLogger('agent')

def traced_execute_tool(tool_name: str, tool_input: dict) -> Any:
    t0 = time.time()
    result = execute_tool(tool_name, tool_input)
    logger.info('tool_call', extra={
        'tool': tool_name,
        'input': tool_input,
        'output_preview': str(result)[:200],
        'duration_ms': int((time.time() - t0) * 1000),
    })
    return result

Structured logs enable you to replay agent sessions, identify which tools cause the most errors, and measure token costs per agent run.

Common Mistakes and Bad Instincts

Building an agent when a pipeline would work. If the sequence of steps is known in advance, use a scripted workflow with LLM-powered steps. Agents add nondeterminism without adding value when the control flow is fixed.

No maximum step limit. An agent without a step limit will loop indefinitely on a confused task or a tool that always returns errors. Always set max_steps.

Not logging tool calls. Without traces, debugging a failing agent session is nearly impossible. Log every tool call in production.

Giving the agent too many tools. Every additional tool exponentially increases the chance of tool selection errors. Start with 2-3 tools. Add more only when the existing tools provably cannot cover required tasks.

Trusting agent output without validation. An agent that calls run_python and returns a number has not "computed" anything reliably - it may have generated the code, but the code might be subtly wrong. Validate outputs with known tests when possible.

Where to Go Next

• context-and-harness-engineering: design what the model sees and the runtime that validates what it does

• mcp-building-a-tool-server: package your tools as an MCP server any client can call

• agent-evals-and-ai-security: grade agent trajectories and defend against prompt injection

• Phase 4 (MLOps and Production) covers deploying, monitoring, and safely operating agents and LLM systems in production.

• Module 22 (Embeddings and RAG) is the retrieval foundation that most agents use as their primary information tool.

• The standalone post llm-application-engineering covers multi-step LLM pipelines in more detail, including structured output patterns and cost management.