Dumb Core, Smart Edge: Agentic Design

Many agentic systems I've watched fail in production had the same shape: intelligence concentrated at the center, where it was hard to test, replace, or reason about. The orchestrator was doing too much — routing requests, holding domain knowledge, managing memory, selecting tools, and shaping outputs, all at once. When something broke, you couldn't isolate it. When requirements changed, you couldn't surgically update it. The whole thing had to move.

This is not a model quality problem. The teams building these systems weren't using inferior LLMs. They were running into an architectural error recurring enough to be useful to name, with a clear counter-principle.

The principle is this: Dumb Core, Smart Edge. The orchestrator — the central node that sequences work — should be nearly stateless, encoding only control flow. The intelligence belongs at the periphery, in specialist nodes that own their domain boundary and can be reasoned about, tested, and replaced in isolation. This is not just a stylistic preference. It is a useful topology when the system has multiple domains, tools, or responsibilities that need to evolve independently.

Why Intelligence at the Core Fails

The instinct to centralize intelligence is understandable. You have one powerful model. You write one system prompt. You give it all the tools and let it figure things out. In a demo, this works. The model is capable enough to juggle concerns simultaneously, and the happy path looks clean.

Production breaks this in four distinct ways.

First: entanglement. When the orchestrator encodes both routing logic and domain knowledge, changing one requires touching the other. A new tool means rewriting the prompt. A new domain rule means re-testing the routing. The system develops coupling that has no architectural boundary — just a long, fragile string of natural language instructions.

Second: untestability and opacity. You cannot write a unit test for an LLM prompt that does five things at once. You can only run end-to-end evaluations and observe whether the emergent behavior is stable. When an agent misbehaves, you need to know whether the failure was in routing, domain reasoning, tool selection, or output formatting. A smart core conflates all of these — regressions are invisible until production, and then you're debugging a black box with a flashlight.

Third: cost amplification. A smart-core agent sends every token of context — full history, all tool schemas, all domain rules — through the same path, for every step. In a multi-step workflow, this repeatedly pays for context the current step may not need. A dumb core routes to specialists. The specialists receive only what they need. At model-routing economics scale — millions of calls — the economics compound fast.

Dimension	Smart Core (Monolithic)	Dumb Core, Smart Edge
Testability	End-to-end only; regressions invisible until production	Each specialist unit-testable with typed inputs/outputs
Cost per request	Full context and all tool schemas sent through one path	Each specialist gets the context slice relevant to its task
Blast radius	Any prompt edit risks cascading regression across all behaviors	Changes isolated to one specialist; orchestrator untouched
Debuggability	Single opaque prompt; failure source ambiguous	Typed contracts at each edge; failure pinpointed to specific node
Model flexibility	Locked to one model for all tasks	Each specialist can use the model or deterministic function that fits its task
Latency	Single call path, but one large prompt	Extra routing step; specialists can run in parallel when the graph allows it

The Principle Defined

The Principle of Structural Periphery: In agent systems with multiple domains or tool boundaries, the capacity for autonomous judgment should be maximally distributed toward the edges of the graph, leaving the core responsible only for state transitions and routing contracts.

This is not a new idea; it is old systems engineering showing up inside agent graphs. Unix pipes work because each program does one thing and communicates through a simple, stable interface. Microservices work — when they work — for the same reason. The internet's core protocols are deliberately simple; the intelligence lives in the endpoints. Dumb Core, Smart Edge applies this same logic to the agent graph.

The orchestrator's job is precisely scoped: receive a typed input, determine which specialist handles it, dispatch, await a result, and emit a typed output. It encodes no domain knowledge. It holds no persistent memory. It makes no judgment calls about content. It is a state machine — and it should be readable as one.

The specialists, by contrast, are autonomous within their domain boundary. A ResearchSpecialist knows how to query sources, evaluate credibility, and synthesize findings. A CodeReviewSpecialist knows the codebase conventions, the failure modes to check, and how to format its output. Each one can be prompted, fine-tuned, swapped, or evaluated entirely independently. The orchestrator never needs to know what changed.

What "Dumb" Actually Means

"Dumb core" is a precise term, not a pejorative. The orchestrator can still use an LLM for the CLASSIFY_INTENT step — classifying an incoming request into a typed intent is a legitimate use of language understanding. What it must not do is reason about the content of that intent, apply domain heuristics, or make judgment calls about how to handle edge cases within a domain.

A useful test: if you removed all domain-specific vocabulary from the orchestrator's system prompt and replaced it with a generic placeholder, would the routing logic still work? If yes, the core is appropriately dumb. If the routing depends on understanding what "a refund request with a disputed charge" means, domain knowledge has leaked into the core and you have a coupling problem.

The orchestrator's system prompt should read like a traffic controller's manual — rules about flow, priority, and failure handling, with no opinion about the cargo. The specialists' prompts should read like expert briefs — deep, opinionated, and narrow.

This also governs memory architecture. Shared, global memory accessible to the orchestrator is a smell. Each specialist should own its working memory for the duration of its task. Persistent memory — user preferences, prior conversation summaries, learned facts — should be retrieved by specialists on demand, not pre-loaded into the orchestrator's context. The orchestrator passes a session identifier, not a memory dump.

The Replaceability Test

The practical proof of Dumb Core, Smart Edge is what I call the replaceability test: you should be able to swap any specialist node — replacing a prompt-based implementation with a deterministic algorithm, a smaller model, or a different specialist implementation — without modifying the orchestrator or any adjacent specialist. The blast radius should stay localized to that specialist boundary.

If a swap requires changes outside the node being replaced, the system has hidden coupling. The most common source: the orchestrator is parsing or depending on the internal structure of a specialist's output, rather than consuming a typed contract. The fix is always the same — define a Pydantic schema at the edge, validate on emission, and let the orchestrator consume the type, never the raw text.

This is where Causal Engineering enters the picture. When you can replace a node and observe the isolated effect on system behavior, you have a causal handle on the system. You can run controlled experiments: same orchestrator, same adjacent specialists, different specialist B. The delta in output quality is attributable to specialist B alone. This is how you improve an agentic system scientifically rather than through intuition and prayer.

Applying This in Practice

The implementation pattern in LangGraph is direct. Define your orchestrator as a graph with typed state — a TypedDict or Pydantic model that carries only routing metadata, session identifiers, and result slots. Each node in the graph is a specialist function that receives a narrowly scoped input and returns a typed output. The edges encode the control flow. The nodes encode the intelligence.

Here is a minimal dumb-core orchestrator in LangGraph v0.4:

from typing import Literal
from pydantic import BaseModel
from langgraph.graph import StateGraph, END

# --- Typed state: routing metadata only, no domain knowledge ---
class OrchestratorState(BaseModel):
    user_input: str
    intent: str = ""
    specialist_result: str = ""
    session_id: str = ""

# --- Intent classifier (the one LLM call the core is allowed) ---
async def classify_intent(state: OrchestratorState) -> dict:
    # Fast classifier model or deterministic classifier
    intent = await llm_classify(state.user_input)  # returns: "research" | "code_review" | "unknown"
    return {"intent": intent}

# --- Routing edge: pure function over typed state ---
def route_by_intent(state: OrchestratorState) -> Literal["research", "code_review", "fallback"]:
    mapping = {"research": "research", "code_review": "code_review"}
    return mapping.get(state.intent, "fallback")

# --- Specialist nodes (each owns its domain boundary) ---
async def research_specialist(state: OrchestratorState) -> dict:
    result = await run_research_agent(state.user_input, state.session_id)
    return {"specialist_result": result.model_dump_json()}

async def code_review_specialist(state: OrchestratorState) -> dict:
    result = await run_code_review_agent(state.user_input, state.session_id)
    return {"specialist_result": result.model_dump_json()}

# --- Graph assembly ---
graph = StateGraph(OrchestratorState)
graph.add_node("classify", classify_intent)
graph.add_node("research", research_specialist)
graph.add_node("code_review", code_review_specialist)
graph.add_node("fallback", lambda s: {"specialist_result": "I can't help with that."})

graph.set_entry_point("classify")
graph.add_conditional_edges("classify", route_by_intent)
graph.add_edge("research", END)
graph.add_edge("code_review", END)
graph.add_edge("fallback", END)

app = graph.compile()

Notice what the orchestrator does not contain: no domain vocabulary, no tool schemas, no memory retrieval, no output formatting logic. The route_by_intent function is a pure mapping. If you added a new specialist tomorrow — say, legal_review — you'd add one node and one entry in the mapping. The existing specialists are untouched.

Each specialist enforces its contract with a typed output schema:

from pydantic import BaseModel, Field

class SpecialistResult(BaseModel):
    """Contract between any specialist and the orchestrator."""
    answer: str = Field(..., min_length=1)
    confidence: float = Field(..., ge=0.0, le=1.0)
    sources: list[str] = Field(default_factory=list)
    needs_escalation: bool = False

class ResearchResult(SpecialistResult):
    """Extended contract for research specialist."""
    key_findings: list[str] = Field(..., min_length=1)
    contradictions: list[str] = Field(default_factory=list)

# Inside the specialist:
async def run_research_agent(query: str, session_id: str) -> ResearchResult:
    raw = await llm_research(query, session_id)
    return ResearchResult.model_validate(raw)  # Pydantic validates on emission

Concretely, this means:

• The CLASSIFY_INTENT node receives raw user input and returns a typed IntentSchema — nothing more. It does not begin answering the question.
• The DISPATCH edge reads the intent type and routes to the correct specialist. This is a conditional edge in LangGraph — a pure function over the typed state.
• Each specialist node receives only what it needs: the original input, its domain context, and a session ID for memory retrieval. It returns a typed SpecialistResult.
• The EMIT_OUTPUT node formats the result for the user. It is also a specialist — it knows presentation conventions, not domain logic.

The orchestrator graph, when drawn, should look like a clean state machine. If it looks like a tangle of conditional logic with embedded domain terms, the principle has been violated.

Where the Principle Has Limitations

No principle applies without friction. Dumb Core, Smart Edge introduces its own costs, and a builder should know them.

Latency. Multi-hop architectures add round trips. Each specialist invocation is a network call or a model call with its own latency budget. For real-time applications, you need to measure whether the classify-then-dispatch step is acceptable. Often it is, because specialists can run in parallel where the monolithic orchestrator would have run sequentially. But measure first — the Iron Triangle of cost, latency, and quality applies here too.

Over-decomposition. Not every agent needs five specialists. A single-purpose agent with a narrow, stable task domain may be better served by one well-prompted model than by an orchestrator routing to three specialists. The principle applies when complexity justifies it — when you have multiple distinct domains, when replaceability matters, when the system will evolve. Apply it with judgment, not dogma.

Interface overhead. Typed contracts at every edge require upfront schema design. For early-stage prototypes, this can slow iteration. The pragmatic path: sketch the system with loose interfaces, then harden them before the first production deployment. Never harden them after you've accumulated technical debt in the orchestrator.

Coordination complexity. When specialists need to collaborate — when the output of one informs the input of another in non-linear ways — the orchestrator's routing logic grows. This is legitimate complexity, but it should live in the orchestrator's conditional edges, not in the specialists themselves. A specialist that reaches out to another specialist directly has broken the topology.

The teams who will build durable agentic systems are the ones who treat their orchestrators like load-bearing walls: minimal, precise, and never asked to do more than their structural role requires. The intelligence can be extraordinary at the edges. The core just has to hold.

The useful question is simple: if your orchestrator's prompt contains domain vocabulary, who owns correctness when that domain changes?

Related Reading:

• The Agentic Contract: Why a Prompt Is Not a Specification — Typed inputs, outputs, and failure modes between components
• When Validators Matter More Than Better LLMs — Stopping invalid specialist output before it becomes shared state
• APIs Are the New Illiteracy — Reading the interface boundaries an organization commits to operate