The trustworthy-looking citation
A source link exists but does not justify the answer. · PM Drills · 24 · 30 sec–2 min
The trustworthy-looking citation · 30 sec–2 min
Situation
The assistant cites an outdated policy.
The answer is well written and the link works. The current policy contains a different exception.
Think before scrolling
Where can the system fail, and how would you test it?
Distinguish source freshness, retrieval selection, and answer grounding.
How a strong PM thinks
Evaluate finding and answering separately.
Check whether the current document is indexed with the right metadata and whether it was retrieved. Then inspect whether the model's claim is supported by the supplied passage.
Define version ownership, refresh behavior, conflict handling, and no-answer behavior. A visible citation improves inspectability but is not proof of correctness.
Takeaway
Evidence must be applicable and actually supportive.
RAG can fail in the source, retrieval, or generation stage; diagnose the stage before changing the prompt.