NLP Fundamentals
Embeddings
Numerical vector representations of text whose distances capture semantic similarity between pieces of content.
Definition
Embeddings are dense vectors — typically 384 to 3,072 numbers — that represent a piece of text in a high-dimensional space. Two passages with similar meaning produce vectors close together; two unrelated passages produce vectors far apart. Embeddings power semantic search, recommendation, classification, and the retrieval step in retrieval-augmented generation. They are not the same as the model that generates text, though both are produced by neural networks trained on language.
Why it matters
Embeddings are why a modern search box can return a result that does not share any keywords with the query, but means roughly the same thing. They underlie 'related articles' modules, content clustering, duplicate detection, and any system that needs to compare meaning rather than exact wording. For content teams, they enable practical features that were prohibitively complex a few years ago: surfacing the right help article for a customer's free-text question, grouping similar product reviews, or finding gaps in a content library. The key practical point is that embeddings are pre-computed and stored, usually in a vector database. A query is embedded at request time and compared against the stored vectors. Latency and cost are both manageable at scale, which is why retrieval-augmented patterns have become standard.
Examples
Semantic match
Query 'how do I cancel my plan?' matches a document titled 'ending your subscription' even with zero shared keywords, because both embed near the same region of vector space.
Clustering
Embedding 5,000 customer messages and clustering them surfaces themes that no manual tagging scheme had revealed.
Frequently asked
Are embeddings unique to each model?
Yes. An embedding from one model is not directly comparable to an embedding from another; the vector spaces differ.
Can embeddings be reversed back into text?
Not perfectly. Embeddings are lossy compressions; some semantic content can be approximated but not exactly recovered.
Do I need a special database for embeddings?
For more than a few thousand vectors, yes — purpose-built vector stores (or vector extensions to existing databases) make similarity search efficient.
Related terms
Vector Database
A specialised database optimised for storing high-dimensional vectors and retrieving nearest neighbours efficiently.
Retrieval-Augmented Generation (RAG)
A pattern in which a model is given relevant context retrieved from your own sources before generating an answer.
Semantic Search
Search that retrieves results by meaning rather than exact keyword match, typically using embeddings and vector similarity.
Put the concept to work
Open the rewrite engine and apply this principle to a draft of your own.
Try a rewrite