Hallucinations and Uncertainty
Design around the consequences of plausible but unsupported output. · AI Product Management · Lesson 51 · 3 min
Hallucinations and Uncertainty · 3 min
Situation
A citation looks real but does not support the claim.
An assistant provides an answer with a source link. The page exists, but the relevant sentence says something different. The presence of a citation made the answer feel verified.
Fluent language and polished presentation can increase trust beyond what the evidence warrants.
Mental model
Distinguish uncertainty from its consequences.
Models can generate unsupported or incorrect statements. The product question is what happens when a user relies on one.
An imperfect brainstorming suggestion may be easy to discard. An incorrect account action can be difficult to reverse. Evaluate the failure's severity, detectability, frequency, and recovery path.
Example
Constrain the role of the answer.
For a delivery assistant, retrieve the current status from an authoritative service. Let the model explain that status without inventing a guaranteed arrival time.
If the source lacks an estimate, state that limitation and offer the available tracking information. Missing evidence should change the answer, not encourage a plausible completion.
Failure case
A confidence number becomes decoration.
A model saying “92% confident” does not automatically represent a calibrated probability of correctness. Users may treat the number as more scientific than it is.
Use validated uncertainty signals where available and evaluate calibration for the actual task. Otherwise, communicate concrete evidence limits such as missing source data or conflicting records.
PM decision
Reduce the chance and the impact.
Use relevant retrieval, constrained outputs, deterministic checks, and appropriate review. Show supporting evidence where it helps users verify. Make correction and escalation easy.
A small disclaimer does not neutralize a workflow that encourages immediate reliance. Match the interface and action permissions to the reliability you have actually demonstrated.
Remember this
Trust should follow evidence and recoverability.
Design what the product does when it cannot establish an answer. “I don't have enough information” can be a useful outcome when paired with a clear next step.