The Hidden Cost of Cheap Intelligence Is Review Debt
Review Debt is the accumulated cost of cheap model outputs requiring expensive review. The fix is boundary contracts upstream, not validators downstream.
- As inference costs drop toward zero, the bottleneck shifts from generation to review — the scarce resource is judgment capacity, not compute
- Review Debt is the accumulated cost of cheap model outputs that require expensive human or model review not budgeted in the original design
- Adding validators downstream does not fix the problem — it processes every output. The architectural fix is boundary contracts upstream.
- Organizations modeling only the cost of generation will accumulate invisible review debt until it surfaces as a quality crisis
- The Supervisor Pattern — explicit boundary contracts, promotion gates, audit trails — is architecture for managing review debt, not a workaround for it
Inference is getting cheaper. Review is getting more expensive. Most teams are only modeling one of these.
The pitch for cheap models is compelling: GPT-4-class reasoning at one-tenth the cost. Flash models that respond in milliseconds. Open-weight models running on commodity hardware. The generation side of the cost curve is collapsing exactly as predicted.
What nobody budgeted is the review side. Every cheap output that enters a workflow must be checked. Every checked output consumes someone's attention — a human reviewer, a validator model, a quality gate. That attention has a cost, and that cost is not dropping.
The Shape of the Debt
I call this Review Debt: the accumulated cost of cheap model outputs that require expensive review not budgeted in the original system design.
Review Debt accumulates invisibly. The generation cost is on the invoice. The review cost is in the calendar — scattered across the hours a senior engineer spends checking outputs, the meetings where a team discusses whether to trust a result, the incidents where an unchecked output reached production.The parallel to technical debt is exact. Technical debt was invisible until teams started measuring it. Code shipped faster when you skipped tests, but the cost of that speed appeared later as bugs, regressions, and the slow creep of untouchable modules. Review debt follows the same trajectory: cheap models ship output faster, but the cost appears later as review backlogs, quality incidents, and the slow erosion of trust in the system.
Both accumulate when you optimize for speed without accounting for the downstream cost of that speed.
Why "Just Add Validators" Fails
I have watched teams reach for the same intuitive fix: a validator at the end of the pipeline. The model generates, the validator checks, bad outputs get caught. This is the approach most teams reach for first.
The problem is that a validator at the end of a cheap-model pipeline still processes every output. If your cheap model generates 1,000 outputs per hour, your validator must review 1,000 outputs per hour. The validator becomes the bottleneck. If the validator is another model, you have added cost. If the validator is a human, you have added latency and the most expensive form of review debt — the kind that scales with throughput.
This is the Validator Asymmetry Principle working in reverse. Validators are cheaper than generators for improving reliability. But validators are not free, and their cost scales linearly with the volume of outputs they must check.
The architectural fix is not validation downstream — it is boundary contracts upstream. The Supervisor Pattern reduces review debt by preventing cheap models from producing outputs that require expensive review in the first place.
The Supervisor Pattern as Debt Management
The Supervisor Pattern addresses review debt at the architecture level. Three mechanisms:
Boundary contracts. Each cheap model operates within explicit constraints — defined input types, output schemas, and scope limits. A model constrained to classification within a fixed taxonomy cannot produce outputs that require free-form review. The boundary contract converts an open-ended review problem into a closed verification problem.
Promotion gates. Outputs from cheap models do not enter the main workflow directly. They enter a staging area where they must meet defined criteria before promotion. This is not a validator — it is a structural gate that separates the cheap model's output space from the system's trusted state. Only outputs that pass the gate require downstream review.
Audit trails. Every cheap model output carries metadata: what it was asked, what it produced, which gate it passed through, and who or what approved the promotion. When review debt surfaces as an incident, the audit trail reveals where the boundary contract was insufficient — which is the information you need to reduce future debt.
How Review Debt Surfaces
In my experience, review debt does not announce itself. It surfaces through three patterns:
The quality incident. An unchecked output reaches production. A customer sees wrong data. A compliance report contains a superseded standard. The immediate cost is the incident response. The structural cost is the discovery that the review system had a gap — one that has existed since the cheap model was deployed.
The review backlog. A team that validates outputs manually accumulates a backlog as throughput increases. The backlog grows faster than the team can process it. The response is usually to lower the review bar — which trades current backlog for future quality incidents.
The trust erosion. Stakeholders stop trusting the system's outputs. They begin spot-checking everything. The cheap model is still running, still generating, still saving on inference costs. But the effective cost of each output now includes the spot-check — which is often more expensive than running a more capable model would have been.
The Budget Model That Works
The correct budget model for a multi-model system accounts for four costs, not one:
| Cost | What it measures | How it scales |
|---|---|---|
| Generation | Inference cost per output | Drops with cheaper models |
| Validation | Cost of checking each output | Scales linearly with throughput |
| Incident recovery | Cost of fixing unchecked outputs | Spikes unpredictably |
| Trust maintenance | Cost of stakeholder spot-checks | Grows as trust erodes |
Organizations that model only the first row will celebrate their declining inference costs while accumulating invisible debt in the other three. The total cost of the system — generation plus review — may increase even as the per-token price drops.
Cheap inference is not cheap intelligence. Intelligence has a review cost. The organizations that model this correctly will build systems where the review architecture is as deliberate as the generation architecture. The rest will discover review debt the way most organizations discover technical debt — in a crisis.Related Reading:
- The Validator Asymmetry Principle — why validation quality buys reliability more cheaply than generation quality
- The Supervisor Pattern — how to use cheap models without losing control
- Judgment Compression — what happens when human review capacity becomes the bottleneck