Agent Frameworks

Agent frameworks are orchestration contracts, not intelligence

Ask what an agent framework is and you will get a demo. The honest definition is narrower and less glamorous: an agent framework is a set of guarantees about what happens around the model call — what persists when a run crashes, which tools the model may reach, how a human interrupts, what gets recorded, and what it costs to find out why the run failed. The model provides the reasoning; the framework provides the contract that makes the reasoning operationally boring.

The definition matters because the category has become a dumping ground. Model routers (Mastra's model gateway, Pydantic AI's provider strings), protocol servers (anything that speaks MCP), coding harnesses like Claude Code, and hosted agent platforms all get called "agent frameworks" in vendor material, because "framework" is free to claim and "runtime" is not. The miscategorization has consequences. If you evaluate a model router as a framework, you will discover at the first crash that it never promised you a checkpoint. If you evaluate a coding harness as a framework, you will discover that its workflow primitives end where your domain begins. In this dossier I treat a framework as: a library you import, that owns the agent loop, that gives you named primitives for state, tools, control flow, and observability. Everything else in the field — routers, harnesses, protocol servers — gets mentioned where it changes a decision, not counted as a peer.

By that definition the field has consolidated to a handful of real options: LangGraph, Pydantic AI, CrewAI, Microsoft Agent Framework (heir to AutoGen), Mastra, the OpenAI Agents SDK, Google's Agent Development Kit, and Hugging Face's smolagents as the minimal counter-argument. I read each project's README, licence file, or documentation landing page for this dossier — the claims below are theirs, the positions are mine.

The anatomy every production agent stack converges on

Strip the branding and every framework above is assembling the same five responsibilities, because the underlying failure modes demand them:

• The loop — call the model, execute the tools it names, feed results back, decide whether to continue. This is the trivial part, and smolagents proves it: its core agent logic fits in roughly a thousand lines of Python.
• State and durability — checkpoints, suspend-and-resume, replay after failure. The moment a run outlives a request, this stops being optional.
• The tool surface — function tools with typed schemas, MCP server connections, and increasingly, sandboxed code execution.
• Verification — evals, rubric graders, and structured output validation, so "the agent finished" is a measured claim.
• Observability — traces, token accounting, cost per run, because un-instrumented agents are undebuggable agents.

The frameworks differ far less in whether they have these pieces than in which piece they treat as the center of gravity. LangGraph centers the state graph. Pydantic AI centers typing. CrewAI centers the social metaphor. The OpenAI Agents SDK and smolagents center minimalism from opposite ends of the ambition scale. ADK centers deployment. Mastra centers the TypeScript application stack. That center-of-gravity choice is the real product decision, because it determines what the framework makes easy and — more importantly — what it makes awkward.

flowchart TD
    LOOP["Model loop: call tools, read results, decide next step"] --> STATE["State and durability: checkpoints, suspend and resume, replay"]
    STATE --> TOOLS["Tool surface: function tools, MCP servers, code execution"]
    TOOLS --> VERIFY["Verification: evals, graders, structured outputs"]
    VERIFY --> OBS["Observability: traces, token accounting, cost per run"]
    OBS -->|"findings feed back into prompts and skills"| LOOP
    HUMAN["Human gate before irreversible actions"] -.->|"approve or reject"| STATE

Nine frameworks, nine different bets

This is the section that earns the page, so each entry gets what it is, what it is genuinely good at, where it stops, and my position.

LangGraph — "a low-level orchestration framework for building, managing, and deploying long-running, stateful agents," in its own words, inspired by Pregel and Apache Beam. Its genuine strengths are durable execution (runs persist through failures and resume from where they left off), first-class human-in-the-loop interrupts, and both short-term and long-term memory. It is MIT-licensed and battle-tested at companies it names: Klarna, Replit, Elastic. Where it stops: it is deliberately low-level. You are buying a state-machine runtime, and the price is conceptual overhead — you model your agent as nodes and edges whether or not your problem is graph-shaped. The ecosystem gravity is real too: LangSmith sits adjacent for tracing and evals, Deep Agents builds on top for subagent-and-filesystem patterns, and LangSmith Deployment is the blessed hosting path. My position: choose LangGraph when you need durable, interruptible, custom state machines and are willing to pay the abstraction tax. Avoid it when a plain loop would do — the most common LangGraph failure I see is a linear three-step workflow dressed in graph clothing, maintained by a team that now needs to understand graph semantics to change a prompt.

Pydantic AI — the typed entry, from the team behind Pydantic itself (which, they correctly note, is the validation layer under the OpenAI SDK, the Anthropic SDK, LangChain, and most of this ecosystem). The pitch is "typed end to end": structured outputs, typed dependency injection, typed tools, so your IDE and type checker agree with what the model returns. Its newer Harness package composes capabilities — the README shows a complete coding agent assembled from a Coder capability that bundles filesystem access, an allowlisted shell, repo context, planning, and context management, with a web-search capability and a second-opinion advisor snapped on beside it. Durable execution is first-party and co-maintained on Temporal, DBOS, and Prefect, with Restate, Kitaru, and Airflow integrations listed. MIT-licensed. Where it stops: it is opinionated Python; the typing discipline is a feature for product engineers and friction for researchers who think in notebooks. My position: the strongest default for Python product teams, because type-checked tool signatures eliminate an entire class of runtime surprise, and durability attached to engines you may already run (Temporal especially) is the correct architecture. The Logfire observability upsell is present but skippable — instrumentation is OpenTelemetry-native.

CrewAI — the social metaphor: Crews of role-playing agents with goals and backstories, plus Flows, its event-driven, precise-control layer. MIT-licensed, Python 3.10 to 3.13, the largest marketing footprint in the field with a claimed 100,000 developers certified through its courses. The two-layer design is smarter than the crew metaphor gets credit for: Flows give deterministic, event-driven control with secure state management between steps, and Crews nest inside Flows when autonomous collaboration is genuinely needed. The commercial tier, CrewAI AMP, adds a managed control plane with tracing, governance, and enterprise support. Where it stops: role-based crews are wonderful in demos and treacherous in production, because autonomous delegation among personas is exactly the pattern whose token consumption and termination behavior are hardest to bound. My position: CrewAI is the fastest path from zero to a working multi-agent demo, and the JSON-first scaffolding (crewai create crew now generates crew.jsonc and agent files, with the old YAML scaffold behind a --classic flag) shows the team knows its on-ramps matter. For production I would use Flows with surgical Crew insertion, never a pure Crew, and I would treat the certified-developer count as a measure of tutorial reach, not production mileage.

AutoGen — the pioneer, now explicitly in maintenance mode. The README carries a caution box: no new features, community-managed, new users directed to Microsoft Agent Framework, existing users pointed at a migration guide. Code is MIT; documentation is Creative Commons Attribution 4.0. Its layered design (Core message-passing runtime, opinionated AgentChat API, Extensions for providers and code execution) and AutoGen Studio for no-code prototyping shaped the field's imagination — Magentic-One was the reference multi-agent team. My position: do not start anything here in 2026. The value of AutoGen today is historical and archaeological: it proved group-chat orchestration patterns and it produced the migration pressure that produced MAF. If you hold an AutoGen estate, the migration guide is your roadmap and the Studio warning — "not meant to be a production-ready app" — was always the tell.

Microsoft Agent Framework — the successor, MIT-licensed, multi-language (.NET and Python first-class, a Go SDK tracked separately), and per its README now at a production-ready 1.0 with stable APIs and a long-term-support commitment. The feature list is enterprise-checklist-as-architecture: graph-based workflow orchestration with sequential, concurrent, handoff, and group patterns; checkpointing, streaming, human-in-the-loop, and time-travel in workflows; OpenTelemetry built in; middleware for request and response pipelines; declarative YAML agents; agent skills; Foundry-hosted agents in two lines of code; and cross-runtime interoperability via A2A and MCP. Its positioning materials are honest about lineage — migration guides from both AutoGen and Semantic Kernel. Where it stops: the Azure gravity is real. Foundry, Azure OpenAI, and Azure credential chains are the paved road, and the README's own legal note that third-party servers, agents, and models are used "at your own risk" under their own licence terms marks where the support boundary sits. My position: the default choice if your organization is on Azure and .NET, and a serious contender in Python; the binding cost is not licence but cloud topology.

Underneath the brand differences, a run looks the same everywhere — and so does the durability contract the framework owes you when that run outlives its process:

sequenceDiagram
    participant Dev as Developer
    participant App as Agent application
    participant Fw as Framework runtime
    participant M as Model provider
    participant T as Tools and MCP servers

    Dev->>App: Define agents, tools, workflow
    App->>Fw: Start run with durable state
    Fw->>M: Model call with tool schemas
    M->>Fw: Tool call request
    Fw->>T: Execute tool
    T-->>Fw: Tool result
    Fw->>M: Continue with result
    M-->>Fw: Final answer
    Fw->>App: Run complete, checkpoint written
    Note over Fw,T: On crash or restart the run resumes from the last checkpoint, not from zero
    alt verification fails
        App->>Fw: Re-run stage with corrected context
    end

Mastra — the TypeScript bet. A framework for AI applications and agents on a modern TS stack: model routing across 40+ providers, agents, a graph-based workflow engine with .then(), .branch(), .parallel() syntax, suspend-and-resume backed by pluggable storage for indefinite human-in-the-loop pauses, conversation history with retrieval and observational memory, built-in evals and observability, MCP server authoring, and Mastra Studio for local building. Y Combinator W25. Licensing here is the instructive case: dual-licensed, with the core framework Apache 2.0 and any directory named ee/ — the README's example is packages/core/src/auth/ee/ — source-available under a Mastra Enterprise License, free for development and testing but requiring a licence for production use. My position: the most complete TypeScript-native answer, and the one I would start a Node or Next.js product on. The ee/ pattern is not a red flag — it is how sustainable open source now often works — but it is a boundary to audit before you build a feature on auth-adjacent code that may sit behind the enterprise licence.

OpenAI Agents SDK — the minimalist, MIT-licensed, Python and JS/TS. Its core concepts list is deliberately short: agents (model plus instructions, tools, guardrails, handoffs), agents-as-tools and handoffs for delegation, guardrails for input and output validation, sessions for automatic conversation history, and built-in tracing. Around that minimal core it has grown the interesting edges: a SandboxAgent preconfigured to work in a container over long time horizons (local Unix sandbox client, Docker client, or hosted), realtime agents on gpt-realtime-2.1 over WebSocket, and voice pipelines. The README claims provider-agnosticism — OpenAI Responses and Chat Completions APIs plus 100+ other LLMs, with acknowledgements crediting LiteLLM and any-llm for that breadth. My position: the least framework per line of code, which is exactly right when your orchestration needs are handoffs and guardrails and exactly wrong when you need durable execution — durability is not in its primitive set, and you will bolt it on yourself or not have it. Treat "provider-agnostic" as structurally true but experientially OpenAI-shaped: the sandbox, realtime, and tracing surfaces are the smoothest on OpenAI's own stack.

Google ADK — the deployment-first entry, Apache 2.0 licensed (verified in the adk-python repository), and the only framework here spanning five languages: Python, TypeScript, Go, Java, and Kotlin. The documentation leads with "build production agents, not prototypes," and the differentiators it names are real: ADK 2.0's graph workflows weave deterministic code with adaptive reasoning under explicit execution paths, and context management is treated as a designed subsystem — sessions, memory, tool outputs, and artifacts assembled into a structured view with automatic filtering, summarization of older turns, lazy artifact loading, and token tracking, rather than string concatenation until the window overflows. Deployment is the gravitational center: containerize anywhere, or one-command deploy to Cloud Run, GKE, or the Agent Runtime on Google Cloud, inheriting managed auth and Cloud Trace. My position: if your production target is Google Cloud, ADK's deployment path is the shortest in the field and its context management is the most thought-through. The Gemini gravity is real but the framework is explicit about adapters for other and locally running models.

smolagents — Hugging Face's ~1,000-line argument for less framework, Apache 2.0. Its distinguishing idea is the CodeAgent: the model writes its actions as Python code rather than emitting JSON tool-call dictionaries, which their materials say uses roughly 30% fewer steps — and hence roughly 30% fewer model calls — with higher performance on difficult benchmarks, citing two papers. It is model-agnostic through InferenceClientModel, LiteLLM, or OpenAI-compatible endpoints, tool-agnostic across MCP servers, LangChain tools, even Hub Spaces, and it can share agents to the Hub. The security honesty is the part every other vendor should copy: the built-in LocalPythonExecutor "is not a security sandbox" and "must not be used as a security boundary"; real isolation requires E2B, Modal, Blaxel, or Docker. My position: smolagents is the best artifact in the field for understanding what an agent actually is — read agents.py in an afternoon — and a legitimate production choice for code-action workloads with proper sandboxing. It is not an enterprise runtime: durability, governance, and deployment are your problem by design.

The comparison, on the axes that actually decide adoption:

Framework	Language	Licence	Durability story	Orchestration center	Commercial gravity	My one-line position
LangGraph	Python, JS	MIT	Checkpoints, interrupts, resume	State graph	LangSmith + Deployments	Best when the problem is truly a state machine
Pydantic AI	Python	MIT	Temporal, DBOS, Prefect first-party	Typed capabilities	Logfire (OTel-optional)	Default for typed Python product teams
CrewAI	Python	MIT	Flows state management; AMP control plane	Role crews inside event-driven flows	CrewAI AMP Suite	Demo-fastest; productionize with Flows, not Crews
AutoGen	Python, .NET	MIT code, CC-BY-4.0 docs	Frozen at maintenance level	Group-chat patterns (legacy)	None — community-managed	Do not start; migrate to MAF
MS Agent Framework	.NET, Python, Go	MIT	Checkpointing, time-travel workflows	Graph workflow patterns	Foundry / Azure	Azure shops' default; 1.0 stability is real
Mastra	TypeScript	Apache-2.0 + Enterprise ee/	Storage-backed suspend and resume	App-included workflows and agents	Mastra Cloud + enterprise licence	Best TS-native stack; audit ee/ boundaries
OpenAI Agents SDK	Python, JS	MIT	Not in the primitive set	Handoffs and guardrails	OpenAI platform tracing	Minimal and clean; bolt durability on yourself
Google ADK	Python, TS, Go, Java, Kotlin	Apache-2.0	Managed sessions and context; Cloud Run or GKE deploy	Context management plus graphs	Google Cloud Agent Runtime	Shortest path when GCP is the destination
smolagents	Python	Apache-2.0	None by design	Code-action ReAct loop	Sandbox partners (E2B, Modal)	Read it, learn from it, sandbox it, respect it

The consolidation nobody scheduled: AutoGen's exit reset the market

The single most consequential recent fact in this field is printed in an orange badge on the AutoGen README: maintenance mode. The pioneer of group-chat multi-agent orchestration — the framework that filled a thousand conference talks — now receives bug fixes and security patches only, and Microsoft's own README routes every new project to Microsoft Agent Framework, with 1.0 declared production-ready and Semantic Kernel users also handed a migration guide. Whether you liked AutoGen or not, the lesson generalizes: framework choice is a bet on an organization's ten-year attention span, and even Microsoft Research lineages get sunset when consolidation is rational.

Around that center of gravity, three smaller shifts alter decisions. Pydantic AI's Harness move — splitting long-running-work capabilities like memory, subagents, context compaction, and a full coding agent into composable, separately versioned pieces — is the clearest convergence with the harness pattern that coding agents popularized. Mastra's adoption of the ee/ dual-licence pattern formalizes the open-core economics of TypeScript infrastructure. And OpenAI's Agents SDK growing sandbox and realtime agents shows the minimalists responding to demand for long-horizon and voice workloads rather than conceding those to the heavy frameworks. The direction of travel across all of them is the same: frameworks are absorbing durability and verification — the responsibilities that used to require separate infrastructure — because that is where prototype-to-production actually breaks.

The licence is benign; the business model is the constraint

Every framework in this survey is permissively licensed at the core — MIT for LangGraph, Pydantic AI, CrewAI, AutoGen's code, Microsoft Agent Framework, and the OpenAI Agents SDK; Apache 2.0 for Mastra's core, smolagents, and Google ADK. Nobody in this field is AGPL, and nothing is research-only. Read the licences as permission grants and they are almost identical.

Read them as business models and they divide into four shapes, and this is the part that decides adoption and almost never appears in comparisons. First, the observability attach: LangGraph is free and LangSmith is the tracing-and-eval surface it is designed to be debugged with; Pydantic AI is OTel-native but Logfire is the one-line path. The framework is MIT; the visibility into it is a subscription, and agent debugging without tracing is expensive in engineer-hours — which is the point. Second, the control plane: CrewAI's AMP adds governance, deployment, and support on top of the free library; this is the Red Hat shape, and it is the honest answer to "how does CrewAI make money" that the certified-developer marketing obscures. Third, the source-available boundary: Mastra's Enterprise License governs production use of ee/ directories — auth being their own example — so a feature you prototype free may sit behind a licence for production, and the audit belongs at architecture time, not launch time. Fourth, the cloud pull: ADK is Apache 2.0 with a deployment story that inherits Google Cloud managed services; Microsoft Agent Framework is MIT with Foundry hosting two lines away. The licence permits everything; the paved road costs metered infrastructure.

The real cost ledger for any of these, in the order it usually bites: engineer time to learn the framework's mental model (LangGraph's graphs and MAF's workflow types are real learning curves); token burn, which multiplies with agent count — a five-agent Crew that deliberates in group chat can spend an order of magnitude more tokens than a single agent with good tools for the same task; observability subscriptions once you need to debug anything; sandbox and execution infrastructure for code-running agents (the E2B, Modal, Docker tier that smolagents' warning makes mandatory in spirit); and durability infrastructure if you self-host Temporal or equivalent — the Pydantic AI integration is free, the Temporal cluster is not.

Choose by constraint, not by tutorial count

Frame the decision by constraints in this order and most choices make themselves.

Language is the first filter and the least negotiable. Mastra for TypeScript products, the Python majority elsewhere, ADK or MAF if you need JVM or Go coverage — MAF's Go SDK and ADK's five languages are the only answers in the field for polyglot organizations.

Durability is the second. If runs must survive restarts, wait on humans indefinitely, or resume after failures, the shortlist is LangGraph, Pydantic AI (on Temporal, DBOS, or Prefect), MAF, and Mastra. If a request-scoped loop suffices — most internal tools, most demos that become features — the OpenAI Agents SDK or smolagents will teach you more per hour invested than any heavyweight.

Cloud topology is the third. Azure-plus-.NET points at MAF with real force; Google Cloud deployment points at ADK, whose one-command Cloud Run and Agent Runtime paths are the shortest route from laptop to managed infrastructure in this survey.AWS- and on-prem shops have the most freedom, which mostly means they get to optimize on the remaining axes.

Data sensitivity and governance is the fourth. If agents touch regulated data, tracing defaults deserve a privacy review before the first real run — traces contain prompts and tool payloads, and the convenient SaaS endpoints (LangSmith, Logfire, hosted tracing) are egress decisions. The frameworks all support self-hosted or OTel-native alternatives; using them is a configuration you must actually make.

Team shape is the tiebreaker. Type-heavy product engineering loves Pydantic AI. Research and experimentation love smolagents and minimal SDKs. Enterprise platform teams that need workflow governance, versioning, and vendor support land on MAF or CrewAI AMP for reasons that have nothing to do with model-call semantics and everything to do with organizational accountability.

Where these frameworks break in production

Each failure mode below is specific, with its trigger, because vague warnings are how teams talk themselves into the same outage twice.

Autonomous-crew token blowups. Trigger: role-based multi-agent collaboration with no explicit termination or budget conditions — the CrewAI Crew pattern taken at face value. Agents politely hand tasks to each other, refine each other's outputs, and consume model tokens at a rate nobody capped. The fix is architectural, not prompt-level: put deterministic Flows (or MAF workflows, or LangGraph edges) around the collaboration, with spend limits at the provider level as the backstop.

Graph-shaped over-engineering. Trigger: adopting a low-level state-machine framework for a pipeline that is actually linear. Every prompt change now routes through node definitions, state reducers, and edge conditions, and the team's velocity quietly halves. The failure is visible in hindsight as a diff that changed three prompt sentences and seven files. The smolagents counter-argument exists precisely because this happens so often.

State-schema lock-in. Trigger: checkpoint formats are framework-private — LangGraph checkpoints, MAF workflow state, Mastra storage records. You discover the cost when AutoGen users discovered it this year: migration means re-encoding everything the system remembers. The honest mitigation is treating persisted agent state as a versioned interface from day one, even though no framework makes it easy.

The un-sandboxed code executor. Trigger: model-written code executed with smolagents' built-in LocalPythonExecutor, which its own README warns is not a security boundary and "can be bypassed." The failure is arbitrary code execution on your host, and the trigger condition is simply running untrusted-model or untrusted-task output locally. The documented alternatives — E2B, Modal, Blaxel, Docker — are the mandatory tier for anything beyond a toy.

MCP trust assumptions. Trigger: connecting third-party MCP servers to an agent. AutoGen's documentation warns plainly that MCP servers "may execute commands in your local environment or expose sensitive information." Every framework that speaks MCP inherits this; the check that matters is whether your organization reviews MCP server provenance the way it reviews npm dependencies. Mostly, today, it does not.

Refusal-shaped outages in guarded models. Trigger: a model whose safety classifiers can decline mid-run — this dossier's Claude Fable 5 article documents the mechanics — combined with a framework loop that does not handle the refusal stop reason. The run dies or deadlocks, and because the response arrives as HTTP 200, naive error handling never notices. Frameworks with first-class fallback chains absorb this; hand-rolled loops usually do not.

Observability egress surprises. Trigger: enabling default tracing without an egress review. Traces carry prompts, retrieved documents, and tool payloads — sometimes customer data — to a vendor endpoint. The failure is discovered by a compliance team, after launch. Decide the tracing backend before production, not after the first incident.

Version-churn whiplash. Trigger: building on 0.x-era APIs and following community tutorials. CrewAI's scaffold change (JSON-first projects with the classic YAML pattern behind a flag) and the field's rapid pre-1.0 iteration mean that a two-year-old tutorial can generate code that no longer reflects the framework's shape. Pin versions, and prefer the project's own quickstart over whatever ranks first in search.

What remains genuinely unresolved

Three questions have no good answers yet, and each could reshuffle this entire comparison. First, does interoperability defeat lock-in? A2A and MCP are real and MAF advertises both, but protocol compatibility covers tool calls and agent discovery, not state schemas — and state is where lock-in actually lives. If durable state does not become portable, the protocols reduce vendor switching friction at the edges while the center stays proprietary. Second, does durable execution belong in the framework or under it? Pydantic AI's answer — attach to Temporal, DBOS, Prefect, the engines enterprises already operate — is architecturally cleaner than every framework shipping its own checkpoint store, and if that view wins, the durability features of LangGraph, MAF, and Mastra become adapters rather than differentiators. Third, do coding harnesses make agent frameworks redundant for the largest actual workload? Claude Code and its peers are, in effect, pre-built frameworks whose orchestration, memory, and verification are opinionated and fixed; if most "agents" organizations ship turn out to be coding-shaped, the market for general frameworks shrinks to the workflows that are not. None of these resolves this year, and all of them are worth revisiting before any framework decision gets called permanent.

Resources

Framework repositories and documentation

• LangGraph README — positioning, durable execution and interrupt features, MIT badge, Pregel and Apache Beam acknowledgements.
• Pydantic AI README — typed-agents pitch, Harness capabilities, durable execution on Temporal, DBOS, and Prefect, ecosystem claims.
• Pydantic AI licence — MIT, verified from the file.
• CrewAI README — Crews versus Flows, AMP Suite control plane, JSON-first scaffolding, MIT badge, certified-developer claims.
• AutoGen README — maintenance-mode notice, migration guidance, layered architecture, Studio security caveat, MIT code with CC-BY-4.0 documentation.
• Microsoft Agent Framework README — 1.0 production-ready claim, workflow patterns, Foundry hosting, A2A and MCP interop, third-party-systems legal note.
• Microsoft Agent Framework licence — MIT, verified from the file.
• Mastra README — TypeScript stack, workflow engine, suspend-and-resume, dual Apache-2.0 plus Enterprise Licence for ee/ directories.
• OpenAI Agents SDK README — core concepts, sandbox and realtime agents, provider-agnosticism, acknowledgements of LiteLLM and any-llm.
• OpenAI Agents SDK licence — MIT, verified from the file.
• smolagents README — CodeAgent design, ~1,000-line core, code-action efficiency citations, sandbox options, LocalPythonExecutor security warning, Apache-2.0 header.
• Google ADK documentation landing page (served from adk.dev) — five-language support, ADK 2.0 graph workflows, context management design, Google Cloud deployment paths.
• Google ADK Python licence — Apache-2.0, verified from the file.