Prompt and Instruction Design
Make the intended behavior explicit and test changes like product changes. · AI Product Management · Lesson 44 · 3 min
Prompt and Instruction Design · 3 min
Situation
“Be helpful” leaves too much unspecified.
An assistant rewrites a customer email but invents a delivery commitment. The team adds “never hallucinate” to the prompt and hopes the problem is solved.
A vague prohibition does not explain what evidence the assistant may use or what it should do when a requested fact is missing.
Mental model
Specify task, evidence, constraints, and fallback.
Instructions describe the intended behavior. Context provides task-specific information. Examples demonstrate important distinctions. Constraints define boundaries and output requirements.
In an API-backed product, system or developer instructions can express application rules. User requests and retrieved documents should not be treated as having the same authority as those rules.
Example
A more useful instruction.
“Draft a reply using only the supplied order facts. Do not promise a delivery date unless one is present. If the date is missing, explain that it is not confirmed and suggest checking tracking. Keep the draft under 120 words.”
Then test an order with a date, without a date, with conflicting dates, and with irrelevant or malicious text in a customer message.
Failure case
Examples teach the wrong pattern.
If every example ends with a confident answer, the model may learn to answer even when information is missing. Include representative abstention, clarification, and exception cases.
Do not add endless instructions after each failure without checking interactions. Contradictory rules and duplicated examples can make the system harder to reason about.
PM decision
Treat prompts as versioned product behavior.
Record what changed and why, then compare outputs against a held-out test set. Evaluate task success, unsupported claims, clarity, and failure handling.
Some constraints should be enforced by code, permissions, or schemas rather than prose alone. A prompt can request a behavior; it is not a security boundary.
Remember this
A prompt is one component of a system.
Give clear evidence and fallback rules, test realistic cases, and enforce critical constraints outside the model where possible. Better wording cannot compensate for missing authoritative data.