AI confidence shown as fake precision

Problem Context

• The UI displays model output, retrieval ranking, classifier score, extractor score, recommendation rank, generated-answer score, or risk estimate as if it were a calibrated probability.
• The number may come from a raw model logit, internal rank, heuristic score, embedding similarity, confidence token, stale assessment, or population-level metric that does not map to user-facing correctness.
• Users may act on the number by approving, denying, sending, publishing, escalating, triaging, applying an extraction, or trusting a generated answer.
• The product may need uncertainty, threshold, source grounding, warning text, or review controls, but the exact-looking number is doing that work instead.

Selection Rules

• Flag this anti-pattern when an AI or automated surface shows a precise score without calibration scope, decision threshold, freshness, uncertainty reason, or safe next action.
• Flag it when decimals, percentages, color bands, stars, gauges, or ranking scores imply correctness that the system cannot honestly support.
• Use confidence / uncertainty display for the corrected pattern when calibrated reliability, review threshold, reason, stale state, and apply gating can be shown honestly.
• Use source grounding display when users need evidence coverage; confidence is not source proof.
• Use citation display when users need claim-level source inspection instead of a reliability score.
• Use meter only for a read-only bounded scalar value with a meaningful range, not a raw AI certainty score.
• Use warning text when the issue is a known severe consequence before an action rather than uncertain model reliability.
• If confidence cannot be calibrated for this task, show qualitative states such as calibration unavailable, insufficient evidence, out-of-distribution, stale model, or review required.
• If a numeric range is valid, show it as an approximate calibrated range with scope, threshold, and freshness rather than an exact-looking single value.
• Do not let high precision unlock copy, apply, approve, deny, send, or automate without matching threshold rules and audit state.

Required States

• Raw score hidden state where an internal score is withheld from the user-facing decision.
• Calibration unavailable state with qualitative reliability label and review path.
• Calibrated range state with task scope, review threshold, and caveat.
• Insufficient evidence state where no numeric confidence is shown because required inputs are missing.
• Out-of-distribution state where confidence is invalid outside monitored scope.
• Stale score state after model, threshold, source, input, or policy changes.
• Threshold changed state that invalidates an old allow or block decision.
• Ranking score not probability state for retrieval or recommendation lists.
• Bad exact percent, decimal certainty, color-only precision, gauge precision, hidden caveat, confidence as source proof, stale precision, and copied score states.

Interaction Contract

• The interface distinguishes raw internal scores from user-facing calibrated confidence.
• A visible confidence value must name its calibration scope, task population, threshold, freshness, and action consequence.
• Uncalibrated, stale, out-of-distribution, insufficient, or ranking-only scores are replaced with qualitative state and recovery actions.
• Changes to prompt, source scope, input, model version, threshold, or policy invalidate the visible score before users act.
• Copy, export, audit, and handoff preserve calibration caveats or omit raw scores that would mislead downstream users.
• High-impact actions below threshold or outside calibration scope route to review, source verification, collect-more-input, escalation, or fallback.
• Accessible labels state uncertainty meaning in words rather than relying on numeric precision, color, or gauge fill.

Implementation Checklist

• Trace each displayed confidence number to its source: calibrated probability, raw score, rank, similarity, heuristic, model confidence, or stale stored value.
• Define whether the number is valid for the current task, population, model version, input quality, source scope, threshold, and consequence.
• Hide or relabel raw scores that cannot be interpreted as user-facing probability.
• Model calibration unavailable, insufficient evidence, out-of-distribution, stale model, threshold changed, ranking-only, copied-score, and review-required states.
• Place uncertainty reason, threshold, freshness, and gated action near the AI output before any commit control.
• Prevent copy, apply, approve, deny, send, publish, or automation from stripping calibration caveats.
• Test exact percent, decimal score, color-only score, gauge score, stale score, source-proof misuse, mobile layout, keyboard flow, high zoom, and status messaging.

Common Generated-UI Mistakes

• Formatting an uncalibrated model score as 97.42 percent sure.
• Showing 0.873 certainty without saying it is a raw rank, similarity, or heuristic score.
• Using a confidence meter or color band to imply correctness without threshold, calibration, or explanation.
• Keeping old confidence after the user edits input, changes source scope, or the model updates.
• Treating confidence as source evidence, policy approval, legal correctness, or permission.
• Copying the exact-looking score into an audit note without the caveat that made it interpretable.

Critique Questions

• What exactly does this number measure, and can users interpret it as probability of correctness?
• Is the score calibrated for this task, population, model version, data source, threshold, and current input?
• Would a qualitative uncertainty state be more honest than a precise percentage or decimal?
• What changes invalidate the displayed score before the user acts?
• Does the UI separate confidence from source grounding, citations, warning text, policy permission, and severe consequence copy?
• Can the user review, verify sources, collect missing input, escalate, or fallback when confidence is unavailable or below threshold?