Building a Multi-Agent AI Coding System: Architecture, Orchestration, and Design Patterns

Introduction

Tools like Cursor, Claude Code, and Windsurf have changed how developers write code. Behind these tools is a multi-agent system where an AI model does not just generate text. It reads files, searches codebases, runs tests, edits code, and coordinates multiple specialized agents to complete complex tasks.

This post breaks down the architecture of these systems. It covers the core components, how agents coordinate, how tools are designed, how context is managed, and what we know about how real products are built. The research draws from official documentation and technical blog posts published between 2024 and 2026.

The Four-Layer Architecture

Every multi-agent coding system has four fundamental layers [1][2].

Layer 1: Augmented LLM

The foundation is a large language model enhanced with retrieval, tools, and memory. The model can generate search queries, select appropriate tools, and decide what information to keep. This is the "brain" that reasons about the task.

Layer 2: Tool Runner (Agent-Computer Interface)

The tool runner bridges the LLM and the real world. It provides capabilities like file reading, file editing, code search, terminal commands, and LSP queries. The design of these tools matters as much as the model itself. Anthropic reports spending more time optimizing tools than prompts for SWE-bench [1].

Layer 3: Context Manager

The context manager selects what information enters the model's context window. This is the most constrained resource. A typical context window holds 128k to 200k tokens, but a large codebase has millions of tokens. The context manager decides what to load, what to cache, and what to discard.

Layer 4: Orchestration Harness

The harness is the control plane. It manages the agent lifecycle, permissions, tool provisioning, and coordination between agents. Cursor's engineering team states this explicitly: "The harness and the model together determine how good the agent is" [5].

The Orchestrator-Workers Pattern

Anthropic identifies five workflow patterns for agentic systems, from simple to complex [1]:

1. Prompt Chaining: Sequential LLM calls with handoff between steps.
2. Routing: A classifier directs input to specialized handlers.
3. Parallelization: Splitting work into concurrent subtasks (either independent sectioning or repeated voting).
4. Orchestrator-Workers: A central LLM dynamically decomposes tasks and delegates to worker LLMs.
5. Evaluator-Optimizer: One LLM generates, another evaluates in a loop.

The orchestrator-workers pattern is the dominant architecture for coding agents. The reason is simple: in coding tasks, you cannot predict the subtasks in advance. The number of files that need changes and the nature of those changes depend on the specific task [1][3].

Anthropic notes: "The most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns" [2].

How Cursor Evolved Its Architecture

Cursor's multi-agent system went through four iterations [4]:

Iteration 1: Self-coordination. Agents tried to coordinate autonomously. This failed because the behavior was unpredictable.

Iteration 2: Structured roles. Introduced explicit Planner, Executor, Workers, and Judge roles. Better, but still fragile.

Iteration 3: Continuous executor. A persistent executor managing the lifecycle of workers. More reliable but limited in scalability.

Iteration 4: Recursive planners with isolated workers. The final design. A root planner owns the full scope. When it identifies subtasks, it spawns sub-planners that fully own their delegated slice. Workers pick up tasks and drive them to completion in isolation. They are unaware of the larger system. When done, they write a handoff document.

This system peaked at approximately 1,000 commits per hour across 10 million tool calls over one week, running on a single large Linux VM [4].

Claude Code's Architecture

Claude Code operates on a three-phase agentic loop: gather context, take action, verify results [7]. The model decides which tools to call and when to iterate.

Subagents for Context Isolation

Subagents are the primary mechanism for scaling context. Each subagent runs in its own context window with a custom system prompt, specific tool access, and independent permissions [8][19].

The context savings are significant. If a subagent reads 6,100 tokens of files and returns a 420-token summary, the parent agent gets the insight without the token cost. That is a 93% reduction in context consumption [13].

Claude Code's built-in subagents include:

• Explore: A fast, read-only agent for searching codebases. Write and Edit tools are denied.
• Plan: A research agent used during plan mode.
• General: A multi-step worker for complex tasks.

Fork Subagents

Fork subagents inherit the full conversation history and prompt cache from the parent. Since the system prompt and tools are identical, the first request reuses the parent's cache, making forks cheaper than spawning fresh subagents [6].

Tool Design: The Agent-Computer Interface

Tool design is where many teams spend the most engineering time. The key principle: tools should match how the model was trained.

Model-Specific Tool Formats

OpenAI models are trained on patch-based editing. Anthropic models use string replacement. Each model can technically use either format, but giving it the unfamiliar one costs extra reasoning tokens and produces more mistakes. Cursor provisions each model with the format it was trained on [5][12].

Absolute Paths Over Relative Paths

When agents change directories, relative paths break. Anthropic found that switching tools to always require absolute paths eliminated an entire category of errors [1].

LSP Integration

LSP (Language Server Protocol) integration is a major differentiator. Claude Code shipped native LSP support in December 2025, exposing goToDefinition, findReferences, hover, and getDiagnostics as native tools [17].

The performance difference is dramatic:

• LSP queries: approximately 50ms latency
• Grep-based search: up to 45 seconds in large codebases
• Token reduction: from approximately 2,000 to approximately 500 tokens for a 100-file project

Tool Taxonomy

A well-designed coding agent exposes tools across several categories:

Category	Examples	Purpose
File operations	read, write, edit	Direct file manipulation
Search	grep, glob, ast_grep	Codebase navigation
Execution	bash, terminal	Running commands and tests
Intelligence	LSP tools	Semantic code understanding
External	web_search, web_fetch	Research and documentation

Context Management: The Critical Constraint

Context management is the most important engineering challenge in multi-agent coding systems [13].

Dynamic Context Discovery

Cursor pioneered the "dynamic context discovery" pattern. Instead of loading everything upfront, agents pull relevant context on demand. In A/B tests, this reduced total agent tokens by 46.9% when calling MCP tools [12].

The principle: "As models have become better as agents, we've found success by providing fewer details up front, making it easier for the agent to pull relevant context on its own" [12].

Prompt Caching

Anthropic's prompt caching costs only 10% of base input token price for cache hits [14]. The cache has a default 5-minute TTL that refreshes on each use. For longer sessions, an extended 1-hour TTL is available at additional cost.

The system uses a 20-block lookback window to find prior cache entries. This means conversations need fewer than 20 blocks between cache breakpoints to maintain hits [14].

Context Compaction

When the context window fills up, systems use compaction: summarizing older history to free space while preserving recent exchanges and key decisions [13]. CLAUDE.md serves as the durable instruction source that survives compaction because it is re-injected from disk after each compaction event.

Codebase Indexing

Cursor uses Merkle trees and simhash to cache embeddings by chunk content. Unchanged code chunks hit the cache without re-embedding. This cuts time-to-first-query from 4.03 hours to 21 seconds at the 99th percentile for large repositories [20].

Scratchpad Management

Cursor found that scratchpads should be frequently rewritten rather than appended to. Agents should auto-summarize when reaching context limits. Self-reflection and alignment reminders in system prompts prevent drift during long-running sessions [4].

Error Recovery and Verification

Coding agents have a natural advantage for error recovery: code solutions are verifiable through automated tests [1][2].

Test Feedback Loops

Agents run tests and iterate based on results. This creates a natural verification cycle that does not exist in other domains.

Checkpoint Recovery

Claude Code automatically snapshots files before each change. Checkpoints can restore conversation only, code only, or both. This provides a safety net for risky operations [3].

Writer/Reviewer Pattern

A fresh context improves code review since the reviewer is not biased toward code it just wrote [3]. This is a simple but effective form of agent-to-agent communication.

Real-World Comparison

Feature	Claude Code	Cursor	Windsurf
Primary pattern	Orchestrator-workers with subagents	Recursive planners with isolated workers	Dual-agent (planner + actor)
Context isolation	Fork subagents, separate windows	Per-worker repo copies	Fast Context subagent
Tool format	String replacement	Model-specific	grep/read/glob
LSP support	Native (v2.0.74+)	Via editor integration	Via Cascade awareness
Cloud execution	Web VMs, agent teams	Cloud VMs with Temporal	Limited public info
SDK	Agent SDK (programmatic)	TypeScript SDK (IDE/CLI/web/SDK)	No public SDK

Windsurf's Approach

Windsurf's Cascade uses a dual-agent system [10][11]:

• Background planning agent: Continuously refines long-term plans while the primary model takes short-term actions.
• Fast Context subagent: Custom SWE-grep models with reinforcement learning, executing up to 8 parallel tool calls per turn over maximum 4 turns.

Cloud Agent Architecture

Cloud agents require decoupling three components [8]:

1. Agent loop: The orchestration logic. Cursor runs this in Temporal, a workflow engine.
2. Machine state: The VM or container running the code. Managed independently from the agent loop.
3. Conversation state: Streaming and storage. Kept separate for resilience.

Cursor's cloud agents run in isolated VMs with repo cloning, auto-create PR capability, and per-run environment variable injection that is encrypted at rest and deleted after the run [19].

Key Takeaways

1. The harness matters more than the model. The orchestration layer, tool design, and context management are what differentiate products, not the underlying LLM.

2. Start with orchestrator-workers. It is the most proven pattern for coding tasks where subtasks cannot be predicted in advance.

3. Isolate context aggressively. Subagents, fork subagents, and per-worker repo copies prevent the main context from bloating.

4. Design tools for the model you are using. Different models need different tool formats. Match the tool to the model's training.

5. Use dynamic context. Load less upfront, let agents fetch what they need. This reduces token waste and improves accuracy.

6. Verify automatically. Test feedback loops and checkpoints are essential for reliable code generation.

7. Keep it simple. The most successful implementations use simple, composable patterns rather than complex frameworks.

Open Questions

1. How do systems handle concurrent file edits from multiple agents without conflicts?
2. What are the token cost implications of different tool-use patterns over long-running sessions?
3. How does model routing work in practice when different agents need different capabilities?
4. What are the failure modes when agents produce incorrect code, and how can automated recovery handle them?
5. How will the Agent Client Protocol evolve to handle multi-agent coordination?

Sources

[1] Building Effective Agents, Anthropic, 2024-12-19. https://www.anthropic.com/research/building-effective-agents

[2] Building Effective Agents (Engineering), Anthropic, 2024-12-19. https://www.anthropic.com/engineering/building-effective-agents

[3] Claude Code Best Practices, Anthropic, 2025. https://www.anthropic.com/engineering/claude-code-best-practices

[4] Self-Driving Codebases, Cursor, 2026-02-05. https://cursor.com/blog/self-driving-codebases

[5] Continually Improving Agent Harness, Cursor, 2026-04-30. https://cursor.com/blog/continually-improving-agent-harness

[6] Sub-agents Documentation, Claude Code, 2025. https://code.claude.com/docs/en/sub-agents

[7] How Claude Code Works, Claude Code, 2025. https://code.claude.com/docs/en/how-claude-code-works

[8] Cloud Agent Lessons, Cursor, 2026-06-02. https://cursor.com/blog/cloud-agent-lessons

[9] Agent SDK, Claude Code, 2025. https://code.claude.com/docs/en/agent-sdk/agent-loop

[10] Cascade Documentation, Windsurf, 2025. https://docs.windsurf.com/windsurf/cascade

[11] Fast Context, Windsurf, 2025. https://docs.windsurf.com/context-awareness/fast-context

[12] Dynamic Context Discovery, Cursor, 2026-01-06. https://www.cursor.com/blog/dynamic-context-discovery

[13] Context Window Documentation, Claude Code, 2026. https://code.claude.com/docs/en/context-window.md

[14] Prompt Caching, Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching

[15] Text Editor Tool, Anthropic, 2025-07-28. https://docs.anthropic.com/en/docs/build-with-claude/tool-use/text-editor-tool

[16] Agent Client Protocol, PromptLayer, 2025-08-27. https://blog.promptlayer.com/agent-client-protocol-the-lsp-for-ai-coding-agents/

[17] LSP Integration in AI Agents, Medium, 2026-02-22. https://medium.com/@vinodh.thiagarajan/lsp-the-protocol-your-ide-uses-every-day-and-now-your-ai-agent-does-too-19e74ca26ace

[18] Tool Use Overview, Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview

[19] Cursor SDK, Cursor, 2025. https://cursor.com/docs/sdk/typescript

[20] Secure Codebase Indexing, Cursor, 2026-01-27. https://cursor.com/blog/secure-codebase-indexing

[21] Oh My Pi, GitHub, 2026-07-14. https://github.com/can1357/oh-my-pi

[22] LSP, Hooks, and Workflow Design, Medium, 2025-12-23. https://medium.com/@sderosiaux/lsp-hooks-and-workflow-design-what-actually-differentiates-ai-coding-tools-288711fa563b

[23] OpenCode LSP, OpenCode, 2026-07-10. https://opencode.ai/docs/lsp