Attentional Gravity: Dashboards Decide What Gets Optimized

The most dangerous metric in your system is the one on your dashboard.

Not because it is wrong. Because it is visible. Visibility creates gravity. Gravity warps trajectories. This is Goodhart's Law, observability bias, and incentive design showing up inside the monitoring stack: the metrics on the screen become the metrics that matter, regardless of whether they map to the actual problem the system was built to solve.

I call this Attentional Gravity — the tendency for measurable signals to attract disproportionate optimization effort, pulling attention away from harder-to-quantify factors that often determine real-world outcomes.

The Gravity Well Problem

Every monitoring stack creates an implicit hierarchy. Grafana puts latency on the first panel. Datadog ranks alerts by frequency. CloudWatch defaults to CPU and memory. These are not neutral design choices — they are attentional architectures that determine where your engineers spend their time.

Attentional Gravity is Goodhart's Law applied at the infrastructure level: when the dashboard becomes the target, it ceases to be a good dashboard.

The mechanism is straightforward. A metric that is easy to collect gets collected. A metric that is collected gets dashboarded. A metric that is dashboarded gets reviewed in standups. A metric reviewed in standups gets optimized. And a metric that gets optimized becomes — gradually, invisibly — the purpose of the system.

I have watched an ML platform team spend three sprints reducing p95 inference latency from 340ms to 280ms on a system where users waited 4-6 seconds for results to render in the browser. The 60ms savings was invisible to every user. But the Grafana panel was green, the sprint retrospective was triumphant, and nobody asked whether the model's answers were actually correct.

The Measurable Triad vs. the Unmeasurable Triad

In LLM-based systems, attentional gravity creates a specific distortion. Three metrics are trivially easy to instrument:

Measurable (high gravity)	Unmeasurable (low gravity)	Actual impact on user value
p95 latency	Factual accuracy	Latency matters — but only past a threshold
Token count per request	Context relevance	Token count is a cost proxy, not a quality signal
Cost per API call	User trust erosion	Cost optimization without quality floors destroys value

The left column requires one line of instrumentation code. The right column requires golden evaluation datasets, human judgment pipelines, longitudinal user studies, and — most critically — an organizational willingness to invest in signals that do not produce satisfying green dashboards.

The asymmetry is not just technical. It is political. A VP can put "reduced p95 latency by 18%" on a slide. Nobody puts "maintained factual accuracy at previous levels" on a slide because it sounds like standing still, even though standing still on accuracy while scaling 3x is genuinely hard engineering.

How Gravity Compounds

Attentional gravity does not just misallocate effort in the present. It compounds over time through three feedback loops.

Hiring gravity. Teams hire for what they optimize. A team obsessed with latency hires systems engineers. Systems engineers see latency problems. The team becomes structurally incapable of even recognizing accuracy problems, because nobody in the room thinks in those terms. The metric shapes the team, and the team reinforces the metric.

Tooling gravity. Internal tooling follows attention. If latency is king, the team builds better latency profilers, better latency dashboards, better latency alerts. Each tool makes latency even more visible, increasing its gravitational pull further. Meanwhile, the accuracy evaluation pipeline — if one exists — remains a Jupyter notebook someone runs manually before quarterly reviews.

The organizational feedback loop of Attentional Gravity: what you measure shapes who you hire, who you hire shapes what you build, and what you build shapes what you can measure. The gravity well deepens with every cycle.

Narrative gravity. Metrics that are visible become the story the organization tells about itself. "We run inference at sub-200ms" becomes an identity. Identities resist challenge. When someone finally measures accuracy and finds it at 71% on domain-specific queries, the response is not alarm — it is rationalization. "Users don't need perfect answers." "The model is good enough for our use case." The metric-as-identity protects itself.

Escape Velocity

Escaping attentional gravity requires deliberate counter-investment — what I think of as epistemic infrastructure. This is not about adding more dashboards. Adding dashboards increases the total gravitational field. It is about changing what gets measured.

Three interventions that I have seen work in practice:

Golden dataset discipline. Build a curated evaluation set of 200-500 representative queries with human-verified correct answers. Run every model change against it. Make the accuracy score as visible as latency — same dashboard, same standup, same sprint review. The golden dataset is the only reliable antidote to accuracy drift in LLM systems.

Shadow metrics. For every primary KPI on your dashboard, name one unmeasured factor that could silently degrade the KPI's meaning. p95 latency? Shadow metric: percentage of fast responses that are factually wrong. Cost per call? Shadow metric: downstream support tickets caused by bad outputs. Force the conversation about what the dashboard hides.

Rotation of attention. Dedicate one sprint per quarter exclusively to metrics the team currently ignores. Not "20% time" — a full sprint with the same rigor applied to the neglected dimension. This is expensive. It is also the only way to prevent the gravitational lock-in described above. The Expert's Shadow pattern applies here — the things your team is best at can blind you to what you are missing.

Where This Breaks

Attentional gravity is not universally bad. Early-stage systems should optimize for what is measurable. When you have 50 users and no monitoring, adding latency dashboards is not distortion — it is survival. The principle becomes dangerous at scale, when the easy metrics are already optimized and the team keeps squeezing them instead of confronting harder questions.

There is also a trap in the corrective. If you make accuracy too visible, it becomes the new gravity well, and teams start overfitting to evaluation sets while ignoring latency regressions that drive users away before they ever see the improved answers. The Iron Triangle of Inference — quality, speed, cost — does not let you maximize all three. Attentional gravity is not about which vertex to choose. It is about making the choice consciously instead of letting your dashboard make it for you.

The most honest question a team lead can ask in a sprint review: what are we not measuring, and what would change if we did?

The systems that fail slowly are not the ones with bad metrics. They are the ones where the dashboard and reality gradually diverge — where every chart shows green while user trust erodes in channels nobody monitors. Attentional gravity is invisible precisely because the evidence against it is, by definition, the evidence you are not collecting.

Related Reading

• The Expert's Shadow — How attentional gravity shapes expert blind spots
• The Golden Dataset Problem — When attention concentrates on the wrong data, evaluation breaks
• The Iron Triangle of Inference — Attention is the hidden currency in the cost-quality-speed tradeoff