What is an embedding?

An embedding is a piece of text converted into a list of numbers — often several hundred or a few thousand of them. The numbers are not arbitrary: they are arranged so that text with similar meaning ends up numerically close together.

That is the entire trick. Once meaning is geometry, "find me things like this" becomes a distance calculation, and computers are extremely good at distance calculations.

Why this is useful

Keyword search matches strings. Embedding search matches meaning. A search for "how do I cancel my subscription" will surface a document titled "ending your membership" — no shared keywords, but the meanings sit near each other.

This is the machinery underneath:

Semantic search — find documents about a topic, not documents containing a word.
RAG — the retrieval step that decides which documents to hand the model.
Recommendations — "more like this", for text.
Deduplication and clustering — spotting that two things say the same thing differently.

The idea's origin

The insight goes back to word vectors in 2013: train a model to predict a word from its neighbours, and the internal representation it learns turns out to encode meaning. Words used in similar contexts land in similar places. Modern embedding models do this for whole sentences and documents rather than single words, but the underlying bet — that meaning can be inferred from context and stored as coordinates — is the same one.

Where it goes wrong

This is the part the tutorials skip. Embeddings measure similarity, not truth, and the difference matters:

Situation	What goes wrong
"Refund policy" vs "no refunds available"	Nearly identical embeddings. Opposite meanings.
Negation generally	"Is safe" and "is not safe" sit close together.
Exact identifiers	Order numbers and error codes are matched poorly. Use keyword search.
Mixed languages	Only works if the model was trained multilingually.
Numbers and dates	Similarity is not comparison. Embeddings can't tell you 2024 is after 2023.

The negation problem is the one that bites hardest in practice: a retrieval system can confidently hand the model a passage that says the exact opposite of what the user needs, because it is about the right topic.

What to do about it

If you're building search, the honest answer is usually both: embeddings for meaning, keyword matching for identifiers and exact terms, combined. Pure vector search looks magical in a demo and disappoints on real queries containing an invoice number.

What we addedA concrete explanation of why "similar" is not "true", and where embedding-based search reliably fails.

This article was researched and drafted with AI assistance from the sources listed below, then checked and edited by Fiqhro Dedhen before publication. How we work.