Every major fine-tuning API takes the same file format, and the format tells you something about the problem it solves. This page covers what JSONL is and why it won, how much data you actually need, and the dataset mistakes that turn an expensive training run into a model that behaves worse than the one you started with.
JSONL in one line
One complete JSON object per line, no wrapping array, no commas between records. The file as a whole is not valid JSON — each line is. That is the entire design, and everything useful about it follows.
The format
| JSON array | JSONL | |
|---|---|---|
| Memory to read | Whole file | One line |
| Append a record | Rewrite the file | Append a line |
| One corrupt record | File unparseable | Skip that line |
| Count records | Parse everything | wc -l |
| Split for parallel work | Requires parsing | Split on newlines |
| Shell tooling | Awkward | grep, head, sort, shuf |
The memory point is decisive at scale. A 10GB JSON array must be fully parsed before the first record is available; a 10GB JSONL file streams at constant memory. That is why it is the format for training data, log shipping, data warehouse exports and anything else that gets large.
⚠️ Three rules that are easy to break
- No newlines inside a record. Every object must be on exactly one line — escape newlines in strings as
\n, never write them literally. - UTF-8, no BOM. A byte order mark makes the first line fail to parse while every other line works.
- Newline at the end of the file. Some readers silently drop a final line without one.
The training format
For chat fine-tuning, each line is one complete conversation:
Multi-turn examples are supported and are how you teach a conversational pattern rather than a single response shape. The model learns to produce the assistant turns; user and system turns are context.
💡 Keep the system prompt consistent with production
Whatever system prompt appears in training is what the model is tuned to expect. If you train with one and deploy with another, you get behaviour that drifts from what you trained for — often subtly enough that it takes a while to notice.
Either use the exact production system prompt in every example, or omit it entirely and rely on the tuning itself. Mixing prompts across examples is the version that reliably disappoints.
How much data
| Examples | Realistically achieves |
|---|---|
| 10–30 | The API accepts it. Effects are unreliable. |
| 50–100 | Consistent tone, format and structure |
| 200–500 | Reliable adherence to a complex output format |
| 500–2,000 | Genuinely new task behaviour |
| 2,000–10,000 | Domain specialisation |
| 10,000+ | Diminishing returns unless the task is genuinely broad |
These are order-of-magnitude guides, and the honest headline is that quality dominates quantity. A hundred carefully constructed, rigorously consistent examples routinely outperform a thousand scraped ones — because inconsistency in the data becomes inconsistency in the model.
The mistakes that ruin a dataset
Inconsistency
The most damaging and the most common. If some examples answer in prose and others in bullet points, some cite sources and others do not, some are terse and others expansive — the model learns that all of those are acceptable and will pick unpredictably.
Write a style specification before building the dataset, and check every example against it. This is tedious and it is the single highest-return activity in the whole process.
Duplicates and near-duplicates
Duplicates over-weight whatever they contain and waste training budget. Exact duplicates are easy to remove; near-duplicates — the same example with a name changed — are the ones that slip through.
For near-duplicates, embed each example and cluster — anything above about 0.95 cosine similarity is worth reviewing by hand.
Leakage between train and validation
If the same example, or a near-copy, appears in both splits, validation loss looks excellent and tells you nothing. Deduplicate before splitting, not after.
Teaching refusal by accident
A subtle one. If your dataset was built by collecting real conversations, it may contain cases where the assistant declined, deflected or said it could not help. Train on those and you teach the model that declining is a valid response — after which it will decline in situations where you very much want it to answer.
Filter these out deliberately, or replace them with the response you actually wanted.
🚨 Fine-tuning can degrade safety behaviour
Research has repeatedly shown that fine-tuning on narrow datasets can weaken safety training that the base model had — sometimes with data containing nothing harmful at all, purely as a side effect of narrow adaptation.
If your fine-tuned model handles user input, re-test its behaviour on safety-relevant cases after training rather than assuming the base model's properties carried through. Do not assume the tuning only affected what you meant it to affect.
Validate before you spend
Training runs cost money and time. Ten seconds of validation catches most of what wastes them.
The token total is the useful output — it tells you what the run will cost before you start it, and the maximum tells you whether any single example exceeds the context limit and will be silently truncated.
Splitting
Ten percent for validation is a reasonable default. On small datasets, holding out too much starves training — with 100 examples, ten held out is defensible and twenty is not.
One important caveat: if your examples have natural groupings — several conversations from the same customer, several questions about the same document — split by group, not by row. Otherwise near-identical examples land on both sides and validation flatters you.
Fine-tuning or retrieval
| Fine-tuning | RAG | |
|---|---|---|
| Teaches | Behaviour — tone, format, style | Knowledge — facts, documents |
| Updating | Retrain | Update the index |
| Cost to change | High | Near zero |
| Citations | Not possible | Natural |
| Latency | Lower — no retrieval step | Higher |
| Per-request cost | Lower — shorter prompts | Higher |
| Stale data | Yes, immediately | No |
The distinction that resolves most cases: fine-tune for how, retrieve for what. If the model needs to answer in a specific format with a specific voice, fine-tune. If it needs to know your product catalogue, retrieve — a catalogue baked into weights is out of date the day it trains and produces confident, unfixable errors.
Fine-tuning also pays for itself on cost in one specific case: when it lets you replace a long system prompt with tuned behaviour. A 2,000-token instruction block sent on every request is expensive, and moving it into the weights removes it from every future call.
Validating JSON records?
Format and check JSON in your browser, or convert CSV exports into the shape you need — nothing uploaded.
Open the JSON Formatter →Summary
- JSONL is one object per line — streamable, appendable, and resilient to a single bad record.
- No literal newlines in records, UTF-8 without a BOM, trailing newline at EOF.
- 50–100 good examples shift tone and format; new behaviour needs several hundred.
- Consistency beats volume. Write the style spec before the dataset.
- Deduplicate before splitting, and split by group where groups exist.
- Filter out refusals unless you want the model to refuse.
- Re-test safety behaviour after tuning. It does not always survive.
- Fine-tune for how, retrieve for what.
Frequently Asked Questions
What is JSONL?
JSON Lines — one complete JSON object per line, separated by newlines, with no surrounding array and no commas between records. It is not valid JSON as a whole file, and that is the point: each line stands alone, so the file can be streamed, appended to and processed line by line.
Why do fine-tuning APIs use JSONL instead of a JSON array?
Because a JSON array must be fully parsed before any record is usable, which means loading a multi-gigabyte file into memory. JSONL can be read one line at a time at constant memory, appended to without rewriting, and a single corrupt line can be skipped rather than invalidating the file.
How many examples do I need to fine-tune a model?
Most APIs accept 10 as a minimum and that is rarely enough for a useful result. 50 to 100 well-chosen examples can shift tone and format reliably. Teaching genuinely new behaviour usually needs 500 to several thousand. Quality and diversity matter far more than raw count.
Should I fine-tune or use RAG?
Fine-tuning teaches behaviour — tone, format, structure, style of reasoning. RAG supplies knowledge. If the model needs to know facts it was not trained on, use retrieval; fine-tuning on facts is expensive, goes stale immediately, and produces confident errors. Many systems need both, for different reasons.
What is the most common fine-tuning dataset mistake?
Inconsistency. If half the examples answer in one format and half in another, the model learns to be inconsistent — and it will be, unpredictably. A small, rigorously uniform dataset outperforms a larger mixed one almost every time.