Back to Writing

Thinking Notes

The complete stack for building robust agents

Six primary layers plus three auxiliary ones: models, harness orchestration, context engineering, protocols, execution, durable execution, observability & evals, gateway, guardrails. Each layer gets its responsibility, a comparison table of the main frameworks, and a selection verdict — with two minimal robust stacks (TS and Python) at the end.

Between "an agent that runs" and "a robust, fully-featured agent" sits an entire stack. This report takes that stack apart: six primary layers plus three frequently-overlooked auxiliary ones, each answering three questions — what problem does this layer solve, which frameworks dominate it, and how do they differ. All comparisons reflect the public state as of September 2026.

How to read it: start with the stack map, then jump to the layers you are missing. Each layer ends with a marked verdict block — conclusions you can take straight to a decision. The reference answers at the end give two minimal robust stacks, TS and Python.

This report answers one concrete question: what layers does it take to build a robust, fully-featured agent, and how do the frameworks within each layer compare. Six primary layers plus three auxiliary ones that are frequently overlooked — each with its responsibility, a comparison table of the main players, and a selection verdict.

One reading path: start with the stack map below, then jump to whichever layer you are missing. All comparisons reflect the public state as of September 2026.

Models Frontier (GPT / Claude / Gemini) + open source (DeepSeek / Qwen / Llama) Harness / Orchestration Agents SDK · Claude Agent SDK · ADK · Vercel AI SDK · Mastra · PydanticAI · LangGraph Context Engineering AGENTS.md · Skills · Memory · RAG Protocols MCP · A2A · AG-UI / A2UI / UCP Execution Sandboxes E2B / Daytona · Browser / Computer-use Durable Execution Temporal · Inngest · DBOS · Restate · Cloudflare / Vercel Workflows Evals + Gateway + Guardrails Langfuse / LangSmith / OTel · LiteLLM / OpenRouter · permissions & policy

Layer 01

Models — set the ceiling, not the floor

The frontier three (GPT / Claude / Gemini) cover most agent tasks; open source (DeepSeek, Qwen, Llama) fills the cost-sensitive, self-hosted and privacy-sensitive gaps. Models set the ceiling of agent capability — robustness comes from the layers below. Models must be swappable at any time: that is the first requirement of stack design.

Verdict: judge models on three things — long-context stability, tool-call reliability, structured-output compliance. In agent work the biggest gaps between models live in those three, not in general benchmarks.

Layer 02

Harness / Orchestration — the skeleton of the agent

This layer decides how control flow is written: loops, branches, handoffs between agents, state management. "Write an agent" means different things in different frameworks — choosing a framework is choosing a control-flow model.

Main frameworks compared

FrameworkLanguageOrchestration primitiveStrengthsBest for
OpenAI Agents SDKPython / TSHandoffs + a minimal four-primitive coreTiny core, fastest to start; tied to the Responses API and built-in tools (search / computer use)Fast delivery inside the OpenAI ecosystem
Claude Agent SDKTS / PythonAgent loop (the Claude Code kernel)Batteries included: file tools, subagents, permission system, hooks, Skills, context compactionThose who want a proven robust kernel, not a DIY kit
Google ADKPython / JavaHierarchical multi-agent (built-in sequential / parallel / loop workflows)Gemini-native, one-stop hosting on Vertex AI Agent Engine, built-in evalsGCP enterprise stacks
Vercel AI SDKTypeScriptUnified model interface + a light Agent abstractionStreaming UI (useChat), multi-model routing, best-in-class TS full-stack experienceTS full-stack, conversational products
MastraTypeScriptAll-in-one: durable workflows + RAG + memory + evalsThe most complete standalone framework in TS, ships a local playgroundTS teams wanting one vendor
PydanticAIPythonType safety: structured output + dependency injectionSeamless with Pydantic / FastAPI, engineered yet lightTeams already on FastAPI
LangGraphPython / JSGraph / state machine: nodes, edges, shared state, checkpointsThe strongest control flow: loops, branches, human-in-the-loop, resume, time-travel debuggingComplex long-running flows with approvals; the enterprise production mainstay
CrewAIPythonRole-based multi-agent (Crew + Task)Lowest expression cost for multi-agent collaborationRole-divided multi-agent demos and light production

On the JVM

FrameworkOriginPositioning
Spring AISpring official (1.0 GA May 2025)The de facto Java standard: unified ChatClient, @Tool, MCP client+server, OTel observability; building blocks, orchestration is on you
LangChain4jCommunityMore "complete": AI Services, RAG, ChatMemory; official Quarkus binding; graph orchestration via LangGraph4j
Google ADK JavaGoogle officialGA at parity with Python: hierarchical agents, built-in evals, Vertex hosting
Spring AI AlibabaAlibabaDeep Qwen / DashScope integration, graph orchestration, JManus (open general agent); the most active Chinese ecosystem
Semantic KernelMicrosoftHas a Java SDK but in maintenance mode — the successor Agent Framework prioritizes .NET / Python only

From demo to production

Every framework above demos in under 50 lines. Production adds durable state, retries, approvals, budgets and evals. Demos never show any of it. So ask which escape hatches you need first: every harness lets you bring your own state, queue and observability, at very different prices; once that is answered, "which framework is best" usually answers itself.

FrameworkState & durabilityStreamingObservability & testingKnown production pitfalls
OpenAI Agents SDKSession memory only; durability is DIY (pair with Temporal / Inngest)Native eventsPlatform traces for OpenAI calls; DIY elsewhereThin middleware: retries, budgets, approvals are yours; the deepest value sits in OpenAI-hosted tools
Claude Agent SDKSessions with built-in compaction; workflow durability still yoursStructured event streamHooks + OTel exportOpinionated runtime (bash / file tools, sandbox assumptions); cost profile tied to Claude; heavier than the word "SDK" suggests
Google ADKSession service; hosting on Vertex Agent EnginePartialBuilt-in evals + GCP tracingGCP gravity; multi-agent abstractions still early
Vercel AI SDKNone built in — Vercel Workflow adds itBest-in-class TS streamingAI SDK telemetry (OTel)The agent loop is thin; long tasks require Workflow; notable API churn across major versions
MastraDurable workflows + memory built inYesLocal playground + evals built inYoung ecosystem, API churn, small hiring pool
PydanticAIDIY (bring your own stores)YesLogfire + OTel; dependency injection keeps unit tests cleanFew batteries: memory, RAG and workflow are all DIY; smaller community
LangGraphCheckpoints (Postgres / SQLite) + time travelYesDeepest LangSmith integrationBoilerplate; checkpoint storage becomes an ops component; platform upsell; 0.x-era churn history
CrewAIBuilt-in memoryLimitedBasicControl flow past the demo stage is hard; debugging multi-crew runs is painful
Durability first: Agents SDK and Vercel AI SDK need an engine bolted on; LangGraph and ADK carry state; Claude Agent SDK has sessions but not workflow durability.
Observability decides debugging speed: agent bugs are state bugs — prefer harnesses with native OTel / trace export, because retrofitting tracing loses the context.
Upgrade risk: young frameworks (Mastra, LangChain4j) churn; wrap the harness behind your own thin agent interface so a swap is a module change, not a rewrite.
Streaming is a chat requirement, not an agent requirement: user-facing assistants need token streams; background agents need durability and evals more.
OpenAI-bound, ship fast: OpenAI Agents SDK
A proven robust kernel out of the box: Claude Agent SDK
GCP / hierarchical multi-agent: Google ADK
TS full-stack chat products: Vercel AI SDK (plus Workflow in production); one-stop TS: Mastra
Python type-driven / FastAPI teams: PydanticAI
Complex long flows, human-in-the-loop: LangGraph
Java enterprise: Spring AI (DIY orchestration) or LangChain4j (assembled)

Layer 03

Context Engineering — what you feed the model decides what comes out

Since 2025 "context engineering" has replaced prompt engineering as the name of this layer. It covers four things: system prompts and conventions, capability packaging, memory, retrieval.

ComponentRepresentativeNotes
Convention filesAGENTS.mdThe "README for agents", a cross-tool convention; a convention, not a component
Capability packagingAnthropic SkillsProcedures, scripts and docs packaged into discoverable capabilities with progressive disclosure — becoming the de facto distribution standard
MemoryMem0 / Letta / ZepExtract-store-recall pipelines that persist state across sessions; pick by hosting model and recall strategy
RetrievalVector stores / RAG pipelinesMoving from prefetch-style RAG to on-demand retrieval: tool-based just-in-time context often beats stuffing up front
Verdict: the highest ROI layer: for the same model, context quality moves outcomes more than switching models does. Convention files and Skills cost almost nothing; do those two first.

Layer 04

Protocols — three families, not one

This layer defines how components talk. Split by the parties communicating; lumping them together is a common mistake.

FamilyProtocolPartiesStatus
Tool interopMCPagent ↔ tools / data sourcesDe facto standard: all three major vendors have adopted it
Agent interopA2Aagent ↔ agentDonated to the Linux Foundation; standardization ran ahead of demand, modest adoption
Interface interopAG-UI / A2UI / UCPagent ↔ frontend / commerceStreaming UI state sync and agentic commerce, evolving fast through 2026
Verdict: use MCP at the tool layer; do not invent your own tool protocol. The agent-to-agent and interface protocols are still moving — verify the need is real before building.

Layer 05

Execution — where model output becomes action

Two sub-kinds: code-execution sandboxes and browser / computer control. The robustness concerns are entirely different.

Sub-kindRepresentativeNotes
Code sandboxE2B / Daytona / ModalFirecracker-grade isolation, snapshot restore, per-second billing — the only safe way to run model-generated code
Browser / computer controlBrowserbase / Playwright MCP / computer-useWhere production incidents concentrate: selector drift, login state, anti-bot; hosted browsers plus explicit approvals beat raw computer-use
Verdict: any agent that runs generated code needs a sandbox layer. No exceptions. Estimate the failure rate of browser automation before estimating its value.

Layer 06

Durable Execution — the line between robust and toy

Long tasks always hit the same wall: a process dies, a model call times out, a step needs retry, a run pauses for human approval. Durable execution engines persist state so tasks can resume, retry and replay.

EngineFormNotes
TemporalSelf-hosted / cloudThe most mature; code-as-workflow (replay model); heavy — the enterprise default
InngestHosted / self-hostedEvent-driven, function-level step retry; smooth in TS
DBOSLibrary (TS / Py)Durable execution inside Postgres — light, no separate component
RestateSelf-hosted / cloudLightweight runtime with clean durable primitives
Cloudflare WorkflowsInside WorkersStep-level durability inside the CF ecosystem, native with D1 / R2
Vercel WorkflowInside VercelThe natural durability layer for the Vercel AI SDK
LangGraph PlatformHostedLangGraph checkpoints are already durable; Platform adds hosting and deployment
Verdict: one criterion decides it all: "the process died at step 7; can it continue from step 7?" If not, it is a toy.

Layer 07 · Aux

Gateway & Routing — the infrastructure floor for multi-model

Model swappability is a design requirement; this is where it lands: one API, routing by cost / latency / availability, budgets and rate limits.

LiteLLM: self-hosted proxy, 100+ models behind one OpenAI-format API, budgets / limits / fallbacks built in
OpenRouter: hosted, 300+ models behind one API — fast integration and price comparison

Layer 08 · Aux

Observability & Evals — wire it in from day one

Agent failures are chained: attributing one detour requires a complete trace. This layer must ship with version one — observability bolted on later never has the context.

ToolFormNotes
LangfuseOpen source, self-hostableTracing + prompt management + evals + datasets; framework-agnostic
LangSmithLangChain commercialDeepest integration with LangGraph
BraintrustCommercialEval-first: datasets, online scoring, experiment comparison
OTel GenAI conventionsStandardVendor-neutral tracing spec; avoids lock-in
Verdict: adopt OTel as the base first, then choose a dashboard. The other order guarantees a rewire.

Layer 09 · Aux

Guardrails & Security — the last line of robust

Three kinds: permission models (tool allowlists, path whitelists, human approval points), output validation (schema checks + fact checks), and data safety (PII filtering, audit logs). Claude Agent SDK's permission system and LangGraph's human-in-the-loop are primitives at this layer.

Verdict: the design principle is not "trust the model" but "assume it will be wrong; cap the blast radius."

Trend

Consolidation — who is eating whom

The stack map is static; the ecosystem is actively merging layers:

Harnesses eat the surroundings: Claude Agent SDK and OpenAI Agents SDK both ship memory, tools and subagents — the standalone memory and tool layers are shrinking
Clouds eat durable execution: Cloudflare and Vercel made Workflow a platform built-in, squeezing standalone engines toward complex scenarios
Standardization kills DIY: after MCP, the reasons to build a proprietary tool protocol have mostly vanished
Evals move left: from post-incident debugging into CI — a PR that fails evals does not merge

Implication: when choosing a framework, read its "absorption roadmap" — the layers you will use in two years are probably already built into today's harness.

Reference answers — two minimal robust stacks

TS stack: Claude Agent SDK (or Vercel AI SDK + Mastra) + MCP tools + Inngest / Vercel Workflow (durability) + E2B (sandbox) + LiteLLM (gateway) + Langfuse (observability).

Python stack: OpenAI Agents SDK or LangGraph + MCP tools + Temporal (durability) + E2B (sandbox) + LiteLLM (gateway) + Langfuse (observability).

Both run in production. They differ only in language and hosting preference. The test for robustness never changes: every layer has to answer "what happens when it breaks."