Every large language model bills by the token, and almost nobody's intuition about tokens is correct. They are not words, not characters, and not syllables. Understanding what they actually are explains a surprising range of otherwise baffling behaviour — including why the same request costs three times as much in one language as another, and why a model that can write a working compiler cannot count the letters in "strawberry".
The working numbers
For ordinary English prose: 1 token ≈ 0.75 words, or about 4 characters. So 1,000 tokens is roughly 750 words. That ratio holds for prose and breaks badly for code, names, numbers and every non-English language.
What a token actually is
A model cannot process text. It processes numbers. Tokenisation is the step that converts one into the other, and the vocabulary it uses is learned from data rather than defined by a rule.
The dominant approach is byte pair encoding. It starts with individual characters and repeatedly merges the most frequently co-occurring pair into a single unit, until the vocabulary reaches a target size — typically 50,000 to 200,000 entries.
The consequence is that common sequences become single tokens and rare ones get split up:
That last point matters more than it looks. Most tokenisers attach the preceding space to a word, so " hello" and "hello" are distinct entries. It is why a prompt ending in a trailing space can subtly change output — you have handed the model a different token sequence than you intended.
💡 Why the vocabulary is a compression trade-off
A larger vocabulary means fewer tokens per document — cheaper and faster — but a bigger embedding table and more parameters spent on rare entries. A smaller vocabulary is more parameter-efficient and splits everything into more pieces.
Providers pick different points on that curve, which is exactly why token counts differ between model families for identical text.
The ratios that actually apply
| Content | Words per token | Chars per token | Relative cost |
|---|---|---|---|
| English prose | 0.75 | ~4.0 | Baseline |
| Technical writing | 0.65 | ~3.6 | +15% |
| Code | 0.45 | ~2.8 | +45% |
| JSON with long keys | 0.35 | ~2.2 | +80% |
| Names and addresses | 0.40 | ~2.5 | +70% |
| Long numbers | — | ~2.0 | +100% |
| Base64 data | — | ~1.5 | +165% |
| Spanish, French, German | 0.55 | ~3.2 | +30% |
| Chinese, Japanese | — | ~1.5 | +150% |
| Hindi, Thai, Burmese | — | ~1.2 | +200% or more |
⚠️ Non-English text costs substantially more
Tokeniser vocabularies are trained on corpora dominated by English, so English words earn dedicated tokens and other languages get split into fragments — sometimes into individual bytes.
The practical effect is that the same meaning, expressed in Hindi or Thai, can cost three to four times what it costs in English, and consumes the context window proportionally faster. It is a real and under-discussed inequity in how these systems are priced, and it is worth measuring rather than assuming if you serve non-English users.
Why the model cannot count letters
The famous failure — asking how many R's are in "strawberry" and getting the wrong answer — is not a reasoning failure. It is a direct consequence of tokenisation.
The model can often answer correctly anyway, because text about spelling appeared in its training data. But it is recalling facts about words rather than inspecting them — which is why it succeeds on common words and fails on unusual ones.
The same mechanism explains several related weaknesses: reversing a string, counting characters, arithmetic on long numbers (which fragment unpredictably), and rhyming in languages where the tokeniser does not align with phonetics.
✅ The fix is to not ask
Character counting, string reversal, exact arithmetic and sorting are all things ordinary code does perfectly and cheaply. Give the model a tool for them rather than asking it to simulate one.
This generalises usefully: any task where the answer is computable should be computed. Reserve the model for tasks that need judgement.
Where the bill actually comes from
Three things routinely make real costs exceed estimates, and none of them is the text you typed.
1. Output costs more than input
Providers bill input and output separately, and output is typically three to five times the input rate. Generation is sequential — each token requires a full forward pass — while input can be processed in parallel.
So a summarisation task, with long input and short output, is cheap. A generation task with a short prompt and a long response is expensive. The intuition that "long prompt = expensive" is backwards for many workloads.
2. Conversations re-send everything
This is the one that surprises people, and it compounds quadratically.
Models are stateless. Every request carries the entire conversation, so a long chat re-bills its whole history on every turn. A twenty-turn conversation costs roughly four times a ten-turn one, not twice.
3. Invisible input
These count as input tokens and you never typed them:
- System prompts — often 500–2,000 tokens, sent every request.
- Tool definitions — each function's schema, sent every request. Ten tools can be 2,000+ tokens.
- Retrieved context — RAG chunks, frequently the largest component.
- Images — billed as tokens, often 1,000+ for a high-resolution image.
- Reasoning tokens — on models that think before answering, billed as output and often invisible in the response.
🚨 Tool definitions are a silent multiplier
Every tool's JSON schema — names, descriptions, parameter types, enum values — is sent on every single request, whether or not the tool is used. An agent with twenty well-documented tools can be paying 4,000 input tokens per turn before the user's message is even considered.
Trim descriptions, remove tools that are not needed for the current task, and check whether your provider supports prompt caching for the static portion. On a high-volume agent this is frequently the single largest cost line.
Estimating before you spend
For anything going to production, use the provider's own tokeniser rather than an estimate — most publish one, and a count is exact where a heuristic is not. Estimate for planning; measure before committing.
Counting words and characters?
Get exact word, character and line counts in your browser — useful for sizing content before you tokenise it.
Open the Word Counter →Reducing token usage
| Technique | Typical saving | Notes |
|---|---|---|
| Prompt caching | Up to 90% on cached input | Biggest single win for repeated system prompts |
| Trim system prompts | 10–30% | Sent on every request — every word compounds |
| Shorten JSON keys | 20–40% on data | ts beats transaction_timestamp |
| Send CSV, not JSON | 40–60% on tables | Keys repeat per row in JSON; headers appear once in CSV |
| Summarise old turns | Bounds growth | Replace early history with a summary |
| Retrieve less | Varies, often large | Fewer, better chunks usually beat more chunks |
| Cap output | Direct | Output is the expensive side |
| Route by difficulty | Large | Send easy requests to a smaller model |
The CSV point is worth spelling out because the saving is so disproportionate. In JSON, every key is repeated for every row:
Over a thousand rows the difference is substantial, and models handle CSV tables perfectly well. Reach for JSON when structure is genuinely nested; use CSV when the data is tabular.
Context windows
The context window is the maximum tokens a model can consider at once — prompt, conversation, retrieved documents and generated output together.
⚠️ A large window is not a reason to fill it
You pay for every token in the window on every request. Latency scales with context length. And retrieval accuracy tends to degrade as context grows — models reliably attend to the beginning and end of a long context and are measurably weaker in the middle.
Sending an entire 200-page document when three relevant pages would do is slower, more expensive, and frequently produces a worse answer. Large windows are a capability for cases that genuinely need them, not a default.
Summary
- 1 token ≈ 0.75 words ≈ 4 characters for English prose, and only for that.
- Code costs ~45% more, JSON ~80% more, CJK and Indic languages 150–200% more.
- Output is billed 3–5× input. Long generations, not long prompts, drive cost.
- Conversations re-send their history, so cost grows quadratically with turns.
- Tool definitions are sent every request and are a common hidden cost.
- Models cannot count letters because they never see them. Use a tool.
- CSV beats JSON by 40–60% for tabular data.
- Do not fill a large context window just because it exists.
Frequently Asked Questions
How many tokens is a word?
For ordinary English prose, roughly 0.75 words per token — so 1,000 tokens is about 750 words. But the ratio is not fixed. Code, rare words, names, numbers and non-English text all use more tokens per unit of meaning, and some languages cost three or four times as much as English for the same content.
Why is my API bill higher than I calculated?
Three usual causes. Input and output are usually billed at different rates, with output several times more expensive. Every message in a conversation is resent on each turn, so a long chat re-bills its whole history each time. And system prompts, tool definitions and retrieved context all count as input even though you never typed them.
Why can't an LLM count letters in a word?
Because it never sees letters. The text is split into tokens before the model receives it, so a word like strawberry may arrive as two or three chunks with no letter-level structure. Asking it to count characters is like asking someone to count the letters in a word they only ever heard spoken in syllables.
Do all models use the same tokeniser?
No, and token counts differ meaningfully between them. The same text can be 15 to 20% more tokens on one model family than another. A count from one provider's tokeniser is an estimate, not a figure, for a different provider.
Does a bigger context window mean I should use it?
Rarely. You pay for every token in the window on every request, latency rises with context length, and retrieval accuracy tends to fall as context grows. Filling a million-token window because it exists is usually slower, more expensive and less accurate than sending only what is relevant.