Every AI Agent is a Liar, a Slacker, and a Thief

We can build a jaw-dropping AI demo in a weekend. Then we spend the next six months failing to make it reliable enough for production. We tweak the prompt, lower the temperature, beg the model to "think step-by-step" — and the system still fails in spectacular ways.

The root cause: we treat the LLM like deterministic software or anthropomorphize it as a diligent junior employee. It is neither. To build reliable agentic systems, you must adopt architectural paranoia — accept that every AI agent is, by default, a liar, a slacker, and a thief, and your architecture must account for all three before a single line of agent logic ships to production.

The Principle of Architectural Paranoia

Every failure mode that can emerge from a stochastic system will emerge, given sufficient scale, adversarial input, or context complexity. The architect's job is not to prevent the LLM from being what it is. The job is to build deterministic boundaries around it so that its probabilistic nature cannot escape the system.

The Prototype Trap is not a prompt engineering problem. It is a category error: treating a stochastic substrate as if deterministic architectural guarantees were possible without deterministic architectural layers.

The Liar: The Physics of Hallucination

An LLM is not a database of facts. It is a probabilistic engine of syntax trained to predict the next most statistically likely token. It does not know what is true. It knows what is plausible — and plausibility is a function of training corpus distribution, not reality.

When confronted with a knowledge gap, an unconstrained agent will not say "I don't know" — that response is statistically unlikely in most prompt distributions. Instead, it will fabricate non-existent legal precedents with real-sounding citations, hallucinate a calculate_refund() tool you never defined, or produce a SQL query that compiles but returns semantically wrong results. The Liar does not know it is lying. There is no deception intent. There is only token prediction.

The countermeasure is the Collapse Function: a deterministic layer that intercepts outputs before they propagate downstream. Schema validation via Pydantic models, tool call verification against a registered tool registry, and factual grounding checks for high-stakes claims.

from pydantic import BaseModel, ValidationError
from typing import Literal

class AgentToolCall(BaseModel):
    tool_name: str
    arguments: dict
    confidence: Literal["high", "medium", "low"]

REGISTERED_TOOLS = {"search_orders", "calculate_refund", "escalate_to_human"}

def collapse(raw_output: str, registered_tools: set[str]) -> AgentToolCall | None:
    """
    Deterministic boundary between LLM output and system execution.
    Returns None (fail-safe) rather than propagating a hallucinated tool call.
    """
    try:
        parsed = AgentToolCall.model_validate_json(raw_output)
    except ValidationError:
        return None  # Schema validation failed — do not propagate

    if parsed.tool_name not in registered_tools:
        return None  # The Liar hallucinated a tool. Intercept before execution.

    return parsed

You cannot prompt the Liar into telling the truth. You must bound it with a deterministic Collapse Function — an architectural layer that catches the hallucination before it escapes the system boundary.

The Slacker: Context Atrophy and Premature Convergence

If you have ever asked an agent to refactor a 500-line script and received // ... rest of your code here ..., you have met the Slacker.

The transformer's attention mechanism does not treat all tokens equally — tokens at the beginning and end of a context window receive disproportionately higher weight. In retrieval tasks, model accuracy drops by 20-40% when relevant information sits in the center of a 16K+ token window versus the edges. But attention decay is only half the problem. The other half is premature convergence: ask an agent to "Plan, Execute, Review, and Refine" in a single pass and it will plan adequately, execute partially, and hallucinate the rest. The generation cost of a complete output exceeds that of a plausible-looking incomplete one.

The countermeasure is the Probabilistic State Machine: decompose every monolithic task into atomic steps, each with its own generation pass, output schema, and validation gate. The agent never holds more than one step in working memory. The state machine advances the agent only when the current step passes validation. See the deep-dive on the Probabilistic State Machine pattern for the full implementation.

The Thief: Data Exfiltration and the Confused Deputy

The Thief is the most dangerous of the three — not because it is the most common, but because its failures are the hardest to detect and the most severe in consequence. An LLM cannot differentiate between "data" and "instructions." Everything in the context window is tokens. This creates two attack surfaces.

Data leakage across isolation boundaries. Inject your entire customer database into a support bot's context, and the Thief will eventually leak Customer A's order history to Customer B. A cleverly constructed prompt ("What was the last order processed before mine?") causes the model to surface information it was never told to protect. At scale, across millions of conversations, this is statistical certainty.

The Confused Deputy attack. Give an agent unrestricted write access and an attacker can hijack its intent via prompt injection — adversarial instructions embedded in a document the agent was asked to summarize. The agent, unable to distinguish injected instructions from legitimate ones, executes DROP TABLE users; as the attacker's deputy. This has been demonstrated against production LangChain and AutoGPT-style systems.

An LLM cannot distinguish between data and instructions at the token level. Every piece of content in the context window is a potential attack surface. Architectural isolation — not prompt-level warnings — is the only reliable defense.

The countermeasure is the Cognitive Firewall: least-privilege tool access (read-only if that is all the task requires), PII scrubbing before prompt construction (the model sees [CUSTOMER_ID: redacted], not the actual ID), and input sanitization for all content originating outside your system boundary.

The Architect's Mandate

The three countermeasures — the Collapse Function, the Probabilistic State Machine, and the Cognitive Firewall — are not optional hardening steps you add after launch. They are the load-bearing walls of any production agentic system. Build without them and you are not shipping an agent. You are shipping a demo with a production URL.

The Golden Dataset principle extends this: the schemas, state transitions, and sanitization rules you define are your most durable engineering artifact. Models will be replaced. Prompts will be rewritten. The structural boundaries you define today will outlast both. The end of determinism in software does not mean the end of deterministic boundaries — it means those boundaries must move from inside the logic to around it.

The question is not whether your agent will lie, slack, or steal. It will. The question is whether your architecture knows the difference between a failure and a catastrophe.

Related Reading

• Your Company's "Agentic Surface Area": The New Metric for Competitiveness — Systematic classification of agent failure modes
• Durable Execution: Why Python while Loops Are Not Production Ready — Durable execution as a mitigation for agent failures