An embedding is a list of numbers that represents meaning. That sentence is accurate and unhelpful, so this page explains what the numbers actually encode, what they reliably capture, what they systematically miss, and the practical arithmetic — storage, cost, and the compression tricks that make a large index affordable.
The idea in one paragraph
An embedding model converts text into a fixed-length list of numbers — a vector — positioned so that texts with similar meanings land near each other. Comparing meaning becomes measuring distance, which computers do extremely fast. That is the whole trick, and everything else is consequences of it.
What the numbers are
Think of each number as a coordinate. Two dimensions give a point on a page; three give a point in a room. Embeddings use hundreds or thousands, which cannot be pictured but behaves the same way — nearby points mean similar things.
No individual dimension means anything interpretable. There is no "formality axis" or "topic dimension" — the representation is distributed across all of them, learned rather than designed. Which is why you cannot inspect a vector to understand why two things matched.
How the positioning is learned
Embedding models are trained with contrastive learning: shown pairs that should be close (a question and its answer, a sentence and its paraphrase) and pairs that should be far apart, and adjusted until the geometry matches.
The training pairs determine what the model considers similar. A model trained on question-answer pairs learns that a question and its answer belong close together even though they share few words. A model trained on paraphrases learns something subtly different. This is why swapping embedding models changes retrieval behaviour in ways that are not always predictable.
Measuring distance
| Measure | What it measures | Range | Use |
|---|---|---|---|
| Cosine similarity | Angle between vectors | −1 to 1 | The default |
| Dot product | Angle and magnitude | Unbounded | Same as cosine if normalised |
| Euclidean distance | Straight-line distance | 0 upward | Occasionally, for clustering |
Cosine similarity is standard because it ignores magnitude. A long document and a short sentence about the same topic point in a similar direction even though one vector is longer — and direction is what carries meaning.
⚠️ Similarity scores are not comparable across models
A cosine score of 0.82 means nothing in isolation. Different models occupy different regions of the space — one may produce scores clustered between 0.7 and 0.95 for everything, another spread across 0.1 to 0.9.
So a fixed relevance threshold tuned for one model is meaningless for another, and "similarity above 0.8" is not a portable rule. Calibrate thresholds empirically per model, or avoid them entirely by taking the top k and reranking.
What embeddings systematically miss
This section matters more than the mechanics, because these are the failures you will actually meet.
Negation
Embeddings capture topical resemblance, and these two sentences are about the same topic in nearly the same words. The single token that reverses the meaning barely moves the vector. Any system retrieving policy text, medical guidance or legal conditions needs to account for this — usually by reranking with a model that reads both texts together.
Numbers and identifiers
Embeddings handle numeric values poorly. "invoice 4471" and "invoice 4472" are nearly identical vectors, and no amount of semantic search reliably distinguishes them. Exact identifiers belong in a keyword index or a database lookup, not a vector search.
Similar is not relevant
The most common practical failure. A question asks how to cancel a subscription; retrieval returns three passages about subscriptions, all topically close, none of which mentions cancellation. They are similar. They are not answers.
✅ Reranking is the standard fix
Retrieve generously — say the top 50 by vector similarity — then pass query and passage together through a cross-encoder reranker, which scores actual relevance rather than resemblance, and keep the top 5.
A cross-encoder reads both texts jointly, so it can tell that a passage is about the right topic and does not answer the question. It is too slow to run over a whole corpus, which is exactly why the two-stage shape works: vectors for cheap recall, reranking for precision.
Adding reranking is frequently the single largest quality improvement available to an existing retrieval system.
Domain mismatch
A model trained on general web text has weaker representations for specialised vocabulary — legal terms of art, clinical abbreviations, internal product names. Two distinct concepts in your domain may sit almost on top of each other because the model never learned they differ.
The storage arithmetic
Straightforward and frequently underestimated.
| Chunks | 768 dims | 1,536 dims | 3,072 dims |
|---|---|---|---|
| 10,000 | 31 MB | 61 MB | 123 MB |
| 100,000 | 307 MB | 614 MB | 1.2 GB |
| 1,000,000 | 3.1 GB | 6.1 GB | 12.3 GB |
| 10,000,000 | 31 GB | 61 GB | 123 GB |
For fast search, vectors generally need to be in memory. At ten million chunks that is a hardware decision, not a configuration one — which is why the compression techniques below matter.
Cutting storage by 30×
Quantisation
Store each value in fewer bits. The accuracy cost is far smaller than intuition suggests.
| Precision | Bytes per value | 1M × 1536 | Typical accuracy retained |
|---|---|---|---|
| float32 | 4 | 6.1 GB | 100% |
| float16 | 2 | 3.1 GB | ~99.9% |
| int8 | 1 | 1.5 GB | ~99% |
| binary | 1/8 | 192 MB | ~90–95% |
Binary quantisation reduces each dimension to a single bit — positive or negative. It sounds destructive and works surprisingly well, because in high dimensions the pattern of signs carries most of the directional information. Similarity becomes a Hamming distance, which hardware computes extremely quickly.
The standard pattern is two-stage: search the binary index to get a candidate set fast, then rescore those candidates with full-precision vectors. Near-full accuracy, a fraction of the memory.
Matryoshka embeddings
Some models are trained so the leading dimensions carry the most information, which means the vector can simply be truncated.
The name comes from Russian nesting dolls — each smaller vector is a complete usable embedding contained within the larger one. Combined with int8 quantisation, a 3,072-dimension index can drop from 12.3GB to under 500MB with most of its quality intact.
Queries and documents are different
A subtlety that materially affects quality.
In symmetric search both sides are the same kind of text — finding duplicate support tickets, say. In asymmetric search a short question is matched against long passages, and the two have very different shapes.
Many embedding models are trained for asymmetric retrieval and expect a prefix marking which side is which:
Omitting the prefix, or using the wrong one, is a quiet quality loss that produces no error. Check the model card — this detail is easy to miss and easy to fix.
Combine with keyword search
Vector search and keyword search fail in complementary ways, which is why the strongest systems use both.
| Vector search | Keyword search (BM25) | |
|---|---|---|
| Paraphrases | Strong | Weak |
| Synonyms | Strong | Weak |
| Exact identifiers | Weak | Strong |
| Rare technical terms | Weak | Strong |
| Numbers | Weak | Strong |
| Negation | Weak | Weak |
Run both and fuse the rankings — reciprocal rank fusion is the usual method and needs no score calibration, which sidesteps the incomparable-scores problem entirely:
Practical notes
- Embed the same way at index and query time. Same model, same version, same prefix. A model upgrade means re-embedding everything.
- Normalise vectors once at index time so search can use the dot product.
- Store the text alongside the vector. You cannot reconstruct text from an embedding, and you need it to return results.
- Batch your embedding calls. Providers charge per token and batching cuts request overhead substantially.
- Cache by content hash. Re-indexing a corpus where 95% is unchanged should not re-embed 95% of it.
- Version your index. When you change model or chunking, build alongside and switch over — not in place.
Working with the data behind your index?
Convert CSV to JSON, format and validate JSON, or hash content for cache keys — all in your browser.
Open the CSV to JSON Converter →Summary
- An embedding positions meaning in space so similarity becomes distance.
- No dimension is individually interpretable. The representation is distributed.
- Cosine similarity measures direction, and scores are not comparable across models.
- Embeddings miss negation, numbers and identifiers. Pair with keyword search.
- Similar is not relevant. Reranking is usually the biggest available win.
- 1M chunks at 1,536 dims is about 6GB before index overhead.
- int8 quantisation cuts 4× at ~99% quality; binary cuts 32× at ~90–95%.
- Check whether your model wants query and passage prefixes.
Frequently Asked Questions
What is an embedding in simple terms?
A list of numbers that represents the meaning of a piece of text, positioned so that texts with similar meanings produce nearby lists. It lets a computer compare meaning by measuring distance, which is what makes semantic search possible without keyword matching.
What does cosine similarity actually measure?
The angle between two vectors, ignoring their lengths. A value of 1 means they point in the same direction, 0 means unrelated and -1 means opposite. It is preferred over straight-line distance because it measures direction — which is what carries meaning — rather than magnitude.
How much storage does a vector index need?
Multiply documents by dimensions by bytes per value. A million chunks at 1,536 dimensions in 32-bit floats is about 6GB before index overhead. Quantising to 8-bit integers cuts that to 1.5GB with minimal accuracy loss, and binary quantisation reaches around 190MB.
Why does semantic search return similar but irrelevant results?
Because similarity is not relevance. An embedding captures overall topical resemblance, so a passage about the same subject scores highly even when it does not answer the question. Embeddings also handle negation poorly — a text saying something is not permitted sits close to one saying it is.
What are Matryoshka embeddings?
Embeddings trained so that the most important information is concentrated in the leading dimensions, which means you can truncate the vector and keep most of its usefulness. A 1,536-dimension vector cut to 512 may retain over 95% of retrieval quality at a third of the storage.