The instinct with a dataset and a language model is to paste the data in and ask a question. That works up to a few hundred rows and then fails β expensively, and often silently, with confidently wrong arithmetic. This page covers the three strategies that scale and the formatting details that cut token cost by more than half.
Pick the right strategy first
Under ~200 rows β send the data. 200 to a few thousand β aggregate or sample. Beyond that β send the schema plus sample rows and have the model write code that runs against the real dataset. The last option scales without limit and is usually more accurate as well.
What a row actually costs
| Row shape | Tokens per row | 1,000 rows |
|---|---|---|
| 5 short columns | ~10 | 10,000 |
| 10 mixed columns | ~22 | 22,000 |
| 10 columns with long decimals | ~40 | 40,000 |
| 10 columns, free text field | ~80 | 80,000 |
| Same data as JSON | ~45 | 45,000 |
Two things stand out. JSON roughly doubles the cost for identical data, because every key is repeated on every row. And unrounded decimals roughly double it again β a detail nobody thinks about that can be the largest single line in the bill.
Formatting that halves the cost
Round every number
Round to the precision the analysis actually needs β usually two decimal places, often zero. If the question is "which region grew fastest", trailing digits are pure cost.
Drop columns you are not asking about
The most effective single reduction, and the easiest to overlook. A table exported from a system typically carries internal IDs, audit timestamps, foreign keys and status flags that have nothing to do with the question.
Normalise dates and shorten headers
Header names are cheap because they appear once. Values are expensive because they repeat. Truncating a timestamp to a date, and using codes instead of full names, pays off on every row.
β Declare units and codes once
Rather than repeating currency symbols or units in every cell, state them in a short preamble:
The preamble costs perhaps 60 tokens once and removes repeated text from every row. It also removes ambiguity the model would otherwise have to guess at.
Sampling
When the model needs to see representative data rather than all of it, which rows you pick matters more than how many.
| Strategy | Good for | Risk |
|---|---|---|
| First N rows | Understanding structure | Sorted data gives a badly skewed view |
| Head + tail | Spotting range and trend | Misses the middle entirely |
| Random | Estimating distributions | Misses rare but important categories |
| Stratified | Most analysis | Needs a sensible strata column |
| Edge cases | Data quality review | Not representative β say so |
β οΈ First-N sampling on sorted data is actively misleading
Exports are usually sorted β by date, by ID, by region. Taking the first 100 rows of a date-sorted export gives the model only the oldest records, and it will describe a pattern that reflects the sort order rather than the data.
Worse, nothing signals the problem. The model produces a confident summary of an unrepresentative slice. Always shuffle or stratify unless you specifically want the earliest rows.
Whatever you send, tell the model it is a sample and how it was drawn. Otherwise it will treat 100 rows as the population and describe them as such.
Aggregating first
Often the model does not need rows at all β it needs the answer to a question about the rows, plus context to interpret it.
This plays to the actual division of strengths. Computing a sum over 50,000 rows is trivial for code and error-prone for a model. Noticing that APAC's refund rate is double EMEA's and suggesting why is the reverse.
The approach that scales without limit
For anything beyond a few thousand rows, send the schema and a sample, and have the model write code that runs against the full dataset.
| Send the data | Send schema, model writes code | |
|---|---|---|
| Row limit | A few hundred | Unlimited |
| Token cost | Scales with rows | Constant |
| Arithmetic accuracy | Unreliable | Exact |
| Reproducible | No | Yes β you keep the query |
| Auditable | No | Yes β you can read it |
| Data leaves your system | Yes | No β only the schema |
That last row matters for anything sensitive. Sending a schema and two anonymised sample rows exposes far less than uploading a customer transaction table, and it is frequently the difference between a workflow that passes review and one that does not.
π‘ Include the data quirks in the schema
Nulls, sentinel values, duplicate keys, mixed date formats, the fact that cancelled rows must be excluded β these are what make generated queries wrong. Stating them in the prompt costs a few tokens and prevents the most common class of error.
A useful habit: keep a short data-notes file alongside each dataset and paste it in. It is documentation you needed anyway.
Fix the data before sending it
| Problem | Effect | Fix |
|---|---|---|
| BOM at file start | First column header unmatched | Strip ο»Ώ |
| Mixed date formats | Model guesses inconsistently | Normalise to ISO |
| Empty vs "N/A" vs "null" | Treated as different values | Pick one and say so |
| Commas inside fields | Column misalignment | Quote properly, or use TSV |
| Newlines inside fields | Rows split | Escape or strip |
| Trailing whitespace | Categories fail to group | Trim every value |
| Inconsistent casing | EMEA β emea | Normalise |
| Duplicate headers | Ambiguous references | Rename |
The BOM is worth singling out because it is invisible and Excel adds it by default. A parser reads the first header as ο»Ώregion rather than region, so that one column silently fails to match while every other column works β which looks like a data problem rather than an encoding one.
π¨ Check what Excel did to your export
If the CSV passed through Excel, assume leading zeros are gone, long identifiers have been rounded to 15 significant digits, and anything resembling a date has been converted. None of it errors and none of it is visible in the file.
Export from the source system directly where you can. Where you cannot, compare a few identifier columns against the original before analysing anything.
Wide versus long
Wide is more token-efficient and reads better when the question compares across columns β "which region declined in April?". Long is better when the question filters or groups, and is the format aggregation code expects.
For sending to a model, wide usually wins: fewer tokens, and the visual layout makes period-over-period comparison easy to see.
Converting between CSV and JSON?
Convert either direction in your browser, with no type guessing and no upload β which matters when the data is customer records.
Open the CSV to JSON Converter βSummary
- Under 200 rows send data; beyond a few thousand send schema and let the model write code.
- CSV costs about half what JSON does for the same table.
- Round every number. Long decimals can double the token count.
- Drop irrelevant columns β usually the single biggest reduction.
- Declare units and codes once in a preamble rather than repeating them per row.
- Never sample the first N rows of sorted data, and always say a sample is a sample.
- Aggregate before sending. Models reason about trends; code computes totals.
- Check for a BOM and Excel damage before trusting the file.
Frequently Asked Questions
How many rows of CSV can I send to an LLM?
As a rough guide, a row of 10 short columns costs 15 to 25 tokens, so 1,000 rows is around 20,000 tokens. That fits most context windows but is expensive and dilutes attention. Beyond a few hundred rows you should be aggregating, sampling, or letting the model write code against the full dataset instead.
What is the best way to analyse a large dataset with an LLM?
Send the schema and a handful of sample rows, then have the model write code β SQL or pandas β that runs against the full data. The model is good at expressing what to compute and poor at being a spreadsheet, so give it the describing job and let ordinary code do the arithmetic.
Does rounding numbers really reduce token usage?
Substantially. Long decimals fragment into several tokens each, so 1234.56789012 can cost twice what 1234.57 costs. Across a table with thousands of numeric cells that is a large share of the total, and the extra precision is almost never used in the analysis.
Should I send CSV or JSON to an LLM?
CSV for anything tabular. JSON repeats every key on every row, so it costs roughly twice as much for identical data and the gap widens with row count. Use JSON only when the data is genuinely nested and cannot be expressed as a flat table.
How do I stop the model miscounting rows or totals?
Do not ask it to count or total anything. Arithmetic over many rows is exactly the task language models are worst at and ordinary code is perfect at. Compute the aggregates yourself and send the results, or have the model write the query and run it separately.