The Most Valuable Page in Your Wiki is the One Titled 'Here Be Dragons'

Every framework tutorial I have ever read ends at the same place: the exact point where production begins. The quickstart.py runs. The demo works. The README says "production-ready". And then you deploy it, and the first thing that happens is something the documentation never mentioned.

I have shipped AI systems into production for a decade. The pattern is always the same. The territory between "it works on my machine" and "it works at 3 AM when the on-call engineer is asleep" is vast, unmapped, and full of dragons. These are the gaps that kill projects — not the architecture, not the model choice, but the things nobody wrote down because they seemed too obvious or too embarrassing to document.

Dragon 1: The Documentation Cliff

Every major agent framework — LangGraph, CrewAI, AutoGen — has documentation that follows the same arc. Setup, hello world, a slightly more complex example, and then a wall of silence where production concerns should be. Connection pooling? Graceful degradation when a tool returns None? Retry semantics when the LLM provider rate-limits you mid-chain? You are on your own.

The cliff is not accidental. Framework authors optimize for adoption, not for the sixth month of operation. The getting-started guide is a sales document. The production guide would be a liability document — it would have to admit every place the abstraction leaks.

The documentation cliff is the gap between a framework's adoption surface and its operational surface — and it is where most production AI projects die.

I have watched teams spend two weeks building a working prototype with LangGraph and then four months discovering everything LangGraph does not tell you about state persistence across process restarts. The prototype is the easy part. The durable execution problem hiding behind it is the dragon.

Dragon 2: Benchmark Theater

Model providers publish benchmarks the way real estate agents photograph apartments — from the most flattering angle, in the best light, with the clutter hidden. MMLU scores, HumanEval pass rates, and arena rankings measure a model's performance under conditions you will never replicate in production.

Your production traffic is not a benchmark. It is a stream of malformed inputs, ambiguous instructions, edge-case prompts that no evaluation suite anticipated, and context windows stuffed to capacity with retrieval noise. The gap between benchmark performance and production reliability is not a percentage — it is a category error. They measure different things.

# What the benchmark tests:
response = client.chat.completions.create(
    model="gpt-5.4",
    messages=[{"role": "user", "content": clean_well_formed_prompt}],
    temperature=0
)

# What production looks like:
response = retry_with_backoff(
    lambda: client.chat.completions.create(
        model="gpt-5.4",
        messages=truncate_to_fit(
            system_prompt      # 2,400 tokens of instructions
            + retrieved_chunks  # 8,000 tokens of noisy retrieval
            + conversation[-6:] # last 6 turns, possibly contradictory
            + user_message,     # often ambiguous or incomplete
            max_tokens=16_384
        ),
        temperature=0.1,
        timeout=30,
    ),
    max_retries=3,
    on_failure=fallback_to_cached_response
)

The second code block is not a more complex version of the first. It is a fundamentally different program. The retry logic, the truncation strategy, the fallback path — none of these appear in benchmark evaluations, and all of them determine whether your system works.

Dragon 3: The Provider Abstraction Lie

The promise of provider abstraction — swap GPT-5.4 for Claude Opus 4.6 with a config change — is technically true and practically false. Every model has behavioral fingerprints that your system learns to depend on. GPT-5.4 handles ambiguous tool schemas differently than Claude Opus 4.6. Gemini 3.1 Pro parses structured output with different failure modes than either. You build implicit contracts with model behavior, and those contracts break silently when you swap providers.

Provider abstraction is an API-level truth and a behavior-level lie — every model swap is a regression test you did not write.

I described why every agent is a liar, a slacker, and a thief, but the deeper problem is that each model lies, slacks, and steals in its own distinctive way. Your error handling is tuned to one model's failure signature. Swap the model, and your error handling catches the wrong things.

Dragon 4: The Recovery Path

Tutorials show the happy path. Production is the recovery path. The question is not "what does this system do when everything works?" but "what does this system do when the LLM returns malformed JSON for the third time in a row, the vector store is returning stale embeddings, and the user just submitted a prompt in a language your system was not trained to handle?"

The iron triangle of inference — latency, cost, quality — becomes a tetrahedron when you add reliability. And reliability is not a feature you bolt on. It is an architectural decision that shapes every other choice. Circuit breakers, graceful degradation, fallback chains, timeout hierarchies — these are the load-bearing walls of production AI, and they do not appear in any quickstart guide.

Recovery paths, not happy paths, are the load-bearing walls of production AI systems.

In our deployments, the ratio of recovery code to feature code in a production agent is roughly 3:1. For every function that does the thing, there are three functions that handle what happens when the thing fails, times out, or returns something unexpected. No tutorial prepares you for this ratio.

Mapping the Territory

Old cartographers marked unknown regions with "here be dragons" — not as decoration, but as honest acknowledgment that their knowledge had limits. The AI engineering field needs the same honesty. We are building on frameworks that are 18 months old, using models that change behavior between versions without changelog entries, deploying into production environments that no benchmark simulates.

The dragons are not going away. Models will keep changing behavior between versions. Frameworks will keep optimizing for adoption over operational maturity. Benchmarks will keep measuring the wrong thing. The only defense is production experience — the kind of knowledge that accumulates in incident reports and on-call runbooks and code comments that say # DO NOT REMOVE — trust me.

The question worth asking is not "how do I avoid the dragons?" but "how fast can I map them?" The teams that survive production are not the ones with the best architecture diagrams. They are the ones with the best except blocks.

Related Reading

• The AI Agent Is a Liar, a Slacker, and a Thief — The dragons are the failure modes no one documented
• The End of Determinism — The map breaks when the territory becomes non-deterministic
• The Iron Triangle of Inference — Dragon territory is where all three triangle constraints bind at once