By Igor Bobriakov in agentic-ai — 08 Jun 2026

Judgment Compression: Who Owns the Expert Decisions Encoded in AI?

When AI ranks several valid options, it executes an organizational policy. Judgment compression works only when its decision rights and accountability are explicit.

An AI system can be structurally correct and still make the wrong organizational decision. A customer reply can satisfy policy, cite accurate account data, and use an acceptable tone while choosing the wrong priority: speed over trust, retention over legal exposure, or consistency over a warranted exception.

Validators cannot settle that choice. They can reject malformed payloads, unsupported labels, prohibited claims, and broken tool calls. They cannot decide which of several valid actions best reflects the organization's risk appetite, customer commitments, or strategic priorities.

I call the mechanism for making those choices repeatable judgment compression: converting diffuse expert preferences and tradeoffs into a compact artifact that an AI system can apply repeatedly. That artifact might be a rubric, scorecard, ranking policy, precedence rule, or preference matrix.

But compression creates a second problem. Once expert judgment becomes executable, it no longer belongs only to the expert who supplied it. It becomes part of the organization's operating model. The decisive questions are therefore not only what the system should prefer, but who may define that preference, approve it, challenge it, update it, and answer for its consequences.

TL;DR — Key Takeaways:

Validation determines whether an option is acceptable; judgment compression determines which acceptable option should win.
A rubric becomes organizational policy when an AI system uses it to allocate attention, risk, service, or authority.
Domain experts should inform the policy, but a named owner must approve it, reviewers must be able to challenge it, and operators must know when to escalate.
Accountability remains with the organization even when the comparison is executed by a model.

The Judgment Layer Is an Operating Model

The Validator Asymmetry Principle establishes an important architectural boundary: reliability does not require a generator to be perfect if outputs can be gated before action. Judgment begins where that gate stops.

Consider an agent drafting a response to a frustrated enterprise customer. Three factually supported candidates pass the policy filter: one is brief and decisive, another expresses uncertainty and requests review, and a third offers a concession intended to preserve the relationship. Choosing among them is not a formatting decision; it allocates risk and expresses what the organization values.

Teams often bury this choice in narrative instructions: be concise but warm; protect trust; avoid unnecessary escalation; prioritize urgency unless uncertainty is material. The model may produce plausible behavior, but the policy remains distributed across prose, examples, reviewer habits, and exceptions. Nobody can easily say which instruction has precedence or who authorized the resulting tradeoff.

A validator asks, "Is this option allowed?" Judgment compression asks, "Which allowed option should win, and who authorized that preference?"

I argue that this is an organizational design problem expressed through AI architecture. The system is not merely generating language. It is executing a decision model. If that model affects customers, employees, spending, access, prioritization, or exposure to risk, its ownership cannot remain implicit.

Judgment sits between validation and action; decision rights determine who controls that layer.

Compression Makes Policy Executable, Not Legitimate

The raw material for judgment is usually scattered: policy documents, arguments between functions, senior operators' instincts, customer commitments, post-incident lessons, and exceptions that have accumulated over time. Compression turns that material into a portable ranking surface.

A useful artifact might say: prefer reversibility over speed; treat factual support as a threshold rather than a weighted preference; escalate when customer impact and uncertainty are both high; never let cost savings override a safety constraint. The precise form matters less than three properties:

Dimensions are explicit. The organization can name the qualities being compared, such as risk, reversibility, trust, latency, cost, or clarity.
Precedence is explicit. The artifact distinguishes hard constraints from preferences and defines what wins when dimensions conflict.
Limits are explicit. The system can identify cases the artifact does not resolve and route them to an authorized reviewer.

Here the context window fallacy becomes operationally dangerous. More instructions do not necessarily create better policy; they can make precedence harder to inspect and failures harder to attribute. Compression should make decision logic smaller, clearer, and easier to contest.

That clarity does not make the policy legitimate by itself. A technically elegant scorecard can still encode one manager's preferences, preserve an obsolete strategy, or conceal a conflict between product, risk, and customer obligations. Extraction and authorization are separate acts: experts can reveal how decisions are made, but the organization must decide whether those decisions should govern the system.

Who Holds the Decision Rights?

Ownership is not the same as authorship. Domain experts may supply the judgment, AI engineers may encode it, and operations teams may observe its effects. None of those activities alone establishes the right to set policy.

Decision right	Typical holder	Required responsibility
Define the outcome	Business or service owner	State what the system is optimizing and which obligations constrain it.
Propose the judgment model	Domain experts with product and operations	Extract recurring comparisons, exceptions, and escalation boundaries.
Encode and test	AI engineering	Implement the approved policy faithfully and expose observable failure modes.
Approve activation	Named accountable owner	Accept the tradeoffs, affected population, operating limits, and residual risk.
Review or challenge	Operations, risk, policy, or another independent function	Inspect outcomes, hear appeals, pause use, and require reconsideration.
Update or retire	Accountable owner through a governed change process	Version changes, assess their effects, communicate them, and preserve rollback.

One person may hold several of these roles in a small organization. The rights still need to be distinguished. Otherwise implementation decisions quietly become policy decisions, reviewers can object without being able to stop deployment, or leaders claim ownership without maintaining the artifact.

AI engineering has a different but equally important obligation. It must preserve the distinction between hard constraints, ranked preferences, and unresolved cases. A system that converts every concern into a weighted score can allow a strong result on one dimension to cancel a failure that should have blocked action entirely. Numerical neatness is not a substitute for policy structure.

The Lifecycle of an Executable Judgment

I would treat a judgment artifact as a governed operational asset rather than a permanent instruction attached to a model.

Extract. Collect recurring comparisons from decisions, policies, disputes, exceptions, and incident reviews. Separate established organizational commitments from individual taste.
Specify. Define dimensions, thresholds, precedence, affected decisions, exclusions, and escalation conditions. Record disagreements instead of averaging them away.
Test. Apply the artifact to representative and adversarial cases. Compare its rankings with authorized human decisions and investigate disagreements rather than treating consensus as automatic truth.
Approve. Name the owner, reviewers, operating scope, review interval, rollback path, and evidence required for expansion.
Operate. Log which policy version influenced a decision, which constraints fired, and whether a human overrode or appealed the result.
Revise or retire. Update the artifact when strategy, obligations, users, evidence, or failure patterns change. Preserve the history needed to explain decisions made under earlier versions.

This lifecycle keeps the judgment layer aligned with the agentic contract, which defines what components may exchange and what an action must satisfy. Judgment compression operates one layer later: it ranks competing options after those conditions are met. Versioning either artifact alone creates a gap between what the system may do and what it is encouraged to do.

Escalation Is Part of the Policy

Human review should not mean asking a person to repeat the same ranking indefinitely. Recurring comparisons that the organization understands should become explicit policy. Humans are most valuable for novel cases, appeals, conflicts between authorities, policy changes, and consequences the current artifact was not authorized to govern.

A useful escalation rule identifies both a trigger and a destination. "Ask a human when uncertain" is incomplete. It should say what kind of uncertainty matters, who has authority to resolve it, what context must accompany the case, and whether the system must pause, choose a reversible action, or continue within a bounded limit.

Each escalation should also be classified. Some reveal missing evidence in a single case. Others expose a defect or gap in the judgment policy itself. If the same issue repeatedly reaches reviewers, the organization should either compress the new judgment into the artifact or declare that the decision remains intentionally human. Otherwise human-in-the-loop becomes a concealed operating dependency rather than a strategic backstop.

What Fails When Ownership Is Unclear

Without explicit rights, organizations accumulate decision debt. Senior operators rescue edge cases, yet their reasoning never becomes shared infrastructure; teams then call contradictory outcomes model unpredictability. Meanwhile, obsolete preferences survive strategy changes because nobody owns their removal.

Several failure modes recur:

Prompt-stuffed policy: important tradeoffs are hidden in prose with no visible precedence or approval history.
Expert capture: one respected reviewer's habits become organizational policy without challenge or authorization.
Orphaned judgment: a rubric remains in production after its author, owner, assumptions, or strategic purpose has disappeared.
Review theater: reviewers can inspect outcomes but lack the authority to pause use, require changes, or hear appeals.
Accountability laundering: leaders describe a consequential choice as "the model's decision" even though the organization selected the data, policy, thresholds, and deployment context.

The decision rule is simple: if an AI system will repeatedly choose among valid options, do not let it act until the organization can name who may approve that preference, who may challenge it, who must update it, and who answers when it causes harm. If those rights cannot be named, keep the system advisory. Compression can transfer the execution of judgment to software; it cannot transfer accountability away from the organization.