The arithmetic of LLM pricing is trivial and almost nobody's first estimate survives contact with production. Not because the sums are hard, but because four multipliers sit between the obvious calculation and the actual bill, and each one is invisible until it arrives.
The formula
Output typically costs 3–5× input. Conversation history is resent every turn. Both facts do more damage to estimates than pricing differences between models.
Input and output are priced separately
This asymmetry is the first thing to internalise, because it inverts where you should look for savings.
A summarisation feature and a drafting feature with similar total token volumes can differ several-fold in cost. Work out which side of the ratio your feature sits on before optimising anything — trimming input on an output-heavy feature is effort spent on the wrong number.
The four multipliers
1. The system prompt, on every single call
A 900-token system prompt is not a one-off. It is 900 tokens per request, forever.
This is also the most tractable of the four. Prompt caching, offered by most providers, charges a reduced rate for a repeated prefix — often a large discount on the cached portion. If your system prompt is stable and substantial, caching it is close to free money.
2. Conversation history, resent every turn
The one that breaks chat budgets. Models are stateless — every turn resends the entire conversation as input.
A twenty-turn conversation can cost ten times a naive twenty-times-one-turn estimate. If your product has long sessions, model this explicitly rather than assuming an average message cost, and consider summarising older turns once a session passes a threshold.
3. Retrieved context
Every retrieved chunk you attach is input you pay for. Retrieving ten chunks of 500 tokens adds 5,000 tokens to every single call, and the common instinct — retrieve more, to be safe — is a direct cost multiplier that often does not improve answers.
4. Retries and failures
Timeouts, rate limits, validation failures and the model producing something unusable all cost full price on the failed attempt. Budget 10–20% for retries; more if you use an evaluator loop, where the retry is the design.
🚨 The multi-step multiplier
An agent or chain multiplies everything above by the number of steps — and it does so on a context that grows with each one:
This is why agent cost is closer to quadratic than linear in step count, and why an agent that occasionally takes sixty steps instead of six is not ten times the cost of a normal run. Cap iterations in code, not in the prompt.
A worked estimate
A support assistant. 5,000 conversations a month, averaging six turns.
Now the instructive part. Two facts jump out of that breakdown, and neither is about model choice:
- Input is 94% of the token volume — but at 3–5× pricing, output is still a meaningful share of the bill. Check both.
- The system prompt and retrieved context are 63% of every request and are identical or near-identical across calls. That is the caching opportunity, and it is larger than any format optimisation.
⚠️ Averages hide the bill
"Six turns average" describes a distribution, and the tail is where the money goes. If 5% of conversations run to forty turns, those 250 conversations can cost more than the other 4,750 combined, because each one is paying quadratic history costs.
Estimate at the median and the 95th percentile. The gap between them tells you whether you need a session length cap — and if the gap is large, you do.
Where the savings actually are
| Lever | Typical saving | Cost to you |
|---|---|---|
| Route easy requests to a small model | 50-80% | A classifier |
| Cache stable prompt prefixes | 30-60% of input | Near zero |
| Retrieve fewer, better chunks | 20-40% | Retrieval tuning |
| Cap output length | 10-30% | Sometimes quality |
| Summarise old conversation turns | Large on long chats | Some fidelity |
| Batch API for non-urgent work | Often ~50% | Latency |
| Change data format | 10-20% | Near zero |
Routing is first by a distance. Most workloads are dominated by requests that a small, cheap model handles perfectly well — classification, extraction, short factual answers, formatting. Sending everything to your most capable model because some requests need it is paying the maximum price for the average request.
Before you build
Four questions, answered in this order, will catch most cost problems while they are still cheap to fix:
- What is the cost of one typical request? Measure it on a real example rather than estimating.
- What is the cost of the worst realistic request? The longest conversation, the largest document, the most agent steps. If you cannot answer, nothing is bounded.
- What is the cost per user per month at expected usage? Compare it to what a user is worth. This is the question that decides whether the feature is viable at all.
- What happens if usage is ten times the forecast? Successful features are the expensive ones.
💡 Instrument from the first day
Providers return exact token usage on every response. Log it per request with the feature name, user and model — it costs nothing and it is the only way to answer "which feature is the bill" later.
Teams that add this after the first surprising invoice spend weeks reconstructing what they could have had for free from day one. Log input_tokens, output_tokens, cached_tokens and the model name, and you can answer any cost question by query rather than by investigation.
For the token estimates that feed all of this, the ratios in tokens vs words vs characters are the practical starting point, and our word counter gives you the exact character counts to run them on.
Sizing prompts and documents?
Get exact character, word and line counts in your browser — nothing is uploaded to a server.
Open Word Counter →Summary
- Output costs 3–5× input. Know which side your feature sits on.
- The system prompt is paid on every call — and is the easiest thing to cache.
- Conversation history is resent each turn, making long chats roughly quadratic.
- Retrieved context is a per-call multiplier, and more chunks rarely means better answers.
- Budget 10–20% for retries, more with evaluator loops.
- Estimate at the median and the 95th percentile. The tail is the bill.
- Routing to a small model is the biggest lever, usually by a wide margin.
- Log token usage per request from day one.
Frequently Asked Questions
How do I estimate LLM API costs?
Take tokens in times the input price, plus tokens out times the output price, multiplied by how many calls you expect. The arithmetic is easy; the accuracy comes from remembering the multipliers — retries, the system prompt on every call, conversation history resent each turn, and any retrieved context you attach.
Why is output more expensive than input?
Because output is generated one token at a time, each requiring a full pass through the model, while input can be processed in parallel. Output pricing is commonly three to five times input pricing, which means a feature producing long responses costs far more than its input volume suggests.
Why does a long conversation get more expensive per message?
Because the whole history is resent with every turn. Message twenty includes messages one through nineteen as input, so cost per message rises as the conversation grows and total cost scales roughly with the square of the conversation length rather than linearly.
What is the cheapest way to reduce LLM costs?
Routing simple requests to a smaller model, which frequently cuts spend by most of the total without measurably affecting quality. After that: caching repeated prefixes, sending less context, and capping output length. Switching data formats helps but is usually a smaller lever than any of these.
How much headroom should I add to an LLM cost estimate?
Budget for two to three times your careful estimate on a first deployment. Real usage distributions have long tails, retries are more common than planned, and the users who adopt a feature most enthusiastically generate far more volume than the median assumption.