When NOT to Use AI
Choose the simplest method that can meet the task's correctness requirements. · AI Product Management · Lesson 55 · 4 min
When NOT to Use AI · 4 min
Situation
The task sounds modern; the rule is already known.
A team wants an LLM to convert spreadsheet columns into a fixed upload format. The source columns and output rules are fully specified.
A deterministic mapping can transform the data consistently, validate it, and explain failures. Adding generation may introduce variability, latency, and cost without solving a real uncertainty.
Mental model
Match the method to the uncertainty.
Use deterministic rules when the decision can be specified reliably. Use traditional software for known workflows and calculations. Use search to retrieve known information. Use analytics to aggregate and compare data.
An LLM can help interpret or generate unstructured language. Human judgment remains important where goals conflict, accountability matters, or evidence is insufficient for automation.
Worked example
Convert the known spreadsheet.
Input columns are customer_id, amount_cents, and currency. The destination requires those same facts with different field names. Write a mapping, validate required fields, and reject malformed rows.
A model should not infer missing amounts or guess currencies. If column names vary unpredictably, it might propose a mapping for review; deterministic code should still perform the approved conversion.
Worked example
Draft questions from messy interview notes.
Here the input is unstructured and several useful outputs are possible. A model can suggest follow-up questions, identify unclear claims, and offer alternative wording.
The output is a draft a researcher can inspect. Evaluate usefulness and unsupported assumptions. The uncertainty is part of the task, so generation has a plausible role.
Compare
Search may be enough.
If users need the exact current return policy, a well-ranked source passage may serve them better than a generated answer. If they need a concise explanation across several applicable sources, generation may add value.
Test the simpler experience first. A chatbot is not automatically the best interface for finding one known fact.
Failure case
Using language generation as a calculator or gatekeeper.
For exact totals, use tested arithmetic on validated data. For a fixed eligibility rule, implement the rule and its exceptions transparently. Let a model explain a verified result if that helps.
Do not place an opaque probabilistic step in a critical path merely because the interface accepts natural language. Natural-language input and deterministic execution can coexist.
Decision
Write the comparison before building.
State the task, acceptable error, baseline method, expected AI benefit, added failure modes, review needs, latency, and ongoing cost. Run a small comparison on representative cases.
If AI does not meaningfully improve the outcome over the simpler baseline, leave it out. If it helps only one uncertain step, isolate that step rather than making the whole workflow agentic.
Remember this
Put uncertainty where it creates value.
Use AI where interpretation or generation is useful or unavoidable. Use deterministic systems where correctness and repeatability can be specified. The strongest product choice can be deciding that no model is needed.