← BackReference (opens in a new tab)

The smaller model

Lower cost per call may increase cost per useful result. · PM Drills · 25 · 30 sec–2 min

The smaller model · 30 sec–2 min

Situation

A cheaper model needs more retries.

The team saves on token charges, but support agents spend longer correcting its drafts. Average completion time also rises.

Think before scrolling

What comparison should decide the model choice?

Include quality, review effort, latency, and the distribution of difficult tasks.

How a strong PM thinks

Compare complete workflows on the same cases.

Measure successful completion, correction time, retries, and total cost. Break results down by task type. A cheap model may fit a narrow routine category even if it is unsuitable for the whole workload.

If routing is proposed, evaluate the router and fallback as well. Do not assume model self-confidence will reliably identify its own errors.

Takeaway

Optimize cost per successful task.

The price of one call is only one component of the product's economics.