"Building Effective AI Agents" is an engineering guide published on the Anthropic engineering blog on December 20, 2024, by Erik S. and Barry Zhang. It is a first-party account of how Anthropic builds agentic systems, and it defines vocabulary used across Anthropic's product documentation: the distinction between workflows and agents, the augmented LLM, and the agent-computer interface (ACI).
Summary of argument
The guide distinguishes two architectural categories. Workflows are systems in which LLMs and tools are orchestrated through predefined code paths; they are described as predictable and consistent, suited to well-defined tasks. Agents are LLMs that dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks; they offer higher flexibility at higher cost and error risk.
Its central recommendation is to "find the simplest solution possible, and only increasing complexity when needed." The guide states that optimizing a single LLM call with retrieval and in-context examples is usually sufficient before building agentic systems.
Building-block patterns
The guide presents a progression of patterns of increasing autonomy:
- Augmented LLM — the foundational unit, combining an LLM with retrieval, tools, and memory. Model Context Protocol (Model Context Protocol (MCP)) is cited as enabling standardized tool integration.
- Prompt chaining — decomposes a task into sequential steps, each LLM call processing the prior output. Suited to tasks cleanly divisible into fixed subtasks where trading latency for accuracy makes sense.
- Routing — classifies input and directs it to specialized handling. Suited to distinct input categories better handled separately, for example different support query types, or routing easy questions to Haiku 4.5 and hard ones to Sonnet 4.6.
- Parallelization — has two variants: sectioning (independent subtasks run simultaneously) and voting (the same task run multiple times for diverse outputs). Suited to multi-faceted analysis where separate LLM calls outperform combined calls.
- Orchestrator-workers — a central LLM dynamically breaks down a task, delegates to worker LLMs, and synthesizes results. It differs from parallelization in that subtasks are not pre-defined. Suited to complex tasks where the number of steps cannot be predicted, such as multi-file code changes.
- Evaluator-optimizer — one LLM generates while another evaluates and provides feedback in a loop. Suited to cases where responses can be demonstrably improved by articulated feedback and the LLM can provide such feedback.
- Autonomous agents — a full loop of plan, act, observe, adjust, repeat. Used for open-ended problems where steps cannot be predicted. The guide states these require extensive sandboxed testing and guardrails because of higher cost and compounding error risk.
Agent-computer interface
The guide argues that tool documentation deserves the same prompt-engineering attention as the overall prompt, and describes this as the agent-computer interface (ACI). Its stated principles are to format tools close to naturally occurring internet text to minimize format overhead; give the model enough tokens to think before it writes itself into a corner; avoid counting overhead such as requiring accurate line counts or string-escaping; use absolute rather than relative filepaths, which the authors report improved reliability in SWE-bench; and treat tool design as writing docstrings for a junior developer.
Documented applications
The guide describes two applications. For customer support, it documents natural conversation flow combined with tool integration over customer data, order history, and refunds, noting that some companies use usage-based pricing that charges only for successful resolutions. For coding agents, it notes that code solutions are verifiable via automated tests so that agents can iterate on test results, and references Anthropic's own SWE-bench Verified implementation.
Key claims
- "The most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns." (high confidence — first-party; consistent with METR findings on scaffolding)
- Frameworks can add complexity that obscures underlying prompts and makes debugging harder; the guide recommends starting with LLM APIs directly. (medium confidence — design philosophy)
- Autonomous agents are best for open-ended problems and require "some level of trust in its decision-making." (high confidence — first-party)
- Agents operating for many turns create higher costs and potential for compounding errors. (high confidence — first-party)
Relevance to existing pages
The workflow/agent distinction and the building-block patterns supply architectural vocabulary used in Agentic AI and are the primary reference for Agent Architecture Patterns. Coding agents are documented as the leading agent deployment use case, connecting the guide to AI Coding Agents through its SWE-bench Verified implementation. The tool-design principles, including sandboxed environments and guardrails, relate to Prompt Injection as partial mitigations for injection attacks.
Relationships
- supports: Agentic AI — provides architectural vocabulary and building blocks
- related: Agent Architecture Patterns — primary source for this concept page
- related: AI Coding Agents — coding agents are the leading deployment use case
- related: Prompt Injection — tool design principles as injection mitigation
- related: Model Context Protocol (MCP) — MCP mentioned as the standard tool integration approach