A vector embedding is the representation of a piece of data — a word, a sentence, a document, an image — as a point in a high-dimensional space, encoded as a list of real numbers. The underlying idea is geometric: if two texts mean similar things, their vectors sit close together in that space; if they mean different things, they sit far apart. That property converts meaning — something fuzzy and linguistic — into distance, something measurable and computable. On top of it, engineers build semantic search, recommendation systems, clustering, and the RAG architecture that gives language models documentary memory. This article explains how embeddings work, how similarity is measured, and how to bring them to production without letting latency or cost spiral out of control.
From words to vectors: what an embedding encodes
Traditional keyword search compares character strings: it finds "car" only if the document literally contains "car," and it fails when faced with "automobile" or "vehicle." Semantic search solves that problem because it operates on meaning, not form. The conceptual leap was popularised by Word2Vec (Mikolov et al., 2013), which demonstrated that semantic relationships are reflected in arithmetic operations on vectors. The famous example — "king − man + woman ≈ queen" — showed that directions in the space capture concepts such as gender or royalty.
Modern embeddings are no longer computed word by word in a static fashion; they are contextual: the same term receives a different vector depending on the sentence in which it appears, thanks to the Transformer architecture introduced in "Attention Is All You Need" (Vaswani et al., 2017) and to subsequent models such as BERT and dedicated embedding model families. As a result, "bank" in "I sat on the bank" and "bank" in "I transferred money to the bank" receive different representations. The typical dimensionality of these vectors ranges from 384 to 3,072 components depending on the model; more dimensions generally capture more nuance at the cost of more storage and compute.
Measuring similarity: cosine, dot product, and Euclidean distance
Once data items are vectors, comparing two of them is a matter of measuring how close they are. The most widely used metric for text is cosine similarity, which measures the angle between two vectors and ignores their magnitude: it equals 1 when the vectors point in the same direction (maximum similarity), 0 when they are orthogonal, and −1 when they are opposite. It is the preferred metric because, in many models, meaning is encoded in the direction of the vector, not its length. The dot product does take magnitude into account and is equivalent to cosine similarity when vectors are normalised. Euclidean distance (L2) measures the straight-line separation between two points and is used when magnitude carries information.
| Metric | What it measures | Typical use |
|---|---|---|
| Cosine similarity | Angle between vectors (ignores magnitude) | Text semantic search |
| Dot product | Magnitude-weighted alignment | Recommenders; equivalent to cosine when normalised |
| Euclidean distance (L2) | Straight-line separation | Clustering when magnitude matters |
Finding the nearest neighbour by comparing a query against every stored vector — exact or brute-force k-NN — is precise but does not scale: with millions of documents, scanning all of them per query is not viable in real time. The solution is Approximate Nearest Neighbour (ANN) search, which trades a small amount of precision for responses in milliseconds. The reference algorithm is HNSW (Hierarchical Navigable Small World), which organises vectors in a layered navigable graph; other families include IVF (inverted file index partitioning) and product quantisation (PQ) for compressing vectors. Vector databases such as FAISS, pgvector, Milvus, and Qdrant implement these indexes.
RAG: giving a language model documentary memory
The application that has driven the surge of interest in embeddings is RAG (Retrieval-Augmented Generation). A language model, however powerful, only knows what it saw during training and is constrained by a context window; it does not know a company's internal documents or information produced after its training cutoff. RAG solves this by retrieving the relevant fragments from a document corpus and handing them to the model as context before it generates a response. The result is an answer grounded in verifiable, proprietary sources, which reduces hallucinations and makes it possible to cite the origin of each claim.
A RAG system operates in two phases. During indexing (offline): documents are split into chunks, an embedding is computed for each chunk, and the embeddings are stored in a vector database. During querying (online): an embedding is computed for the user's question, the k most similar chunks are retrieved, and a prompt is constructed that combines the question with those chunks so the model can generate a grounded response. System quality depends on the embedding model and the chunking strategy just as much as on the generative model itself. A common technique for improving precision is reranking: retrieve many candidates with fast vector search, then reorder them with a more capable model before passing the top results to the generator.
Taking embeddings to production and the regulatory landscape
Beyond RAG, embeddings enable a range of other use cases: clustering to discover topics in large corpora without prior labels, semantic deduplication, anomaly detection, and classification by similarity to reference examples. Bringing any of these to production requires deliberate engineering decisions: which embedding model to use (and whether it is multilingual, which matters for non-English content), how to chunk documents, which similarity metric and ANN index to choose, and how to manage reindexing when documents change or when the model is updated — switching models requires recomputing all vectors, because embeddings from different models are not mutually comparable.
When documents contain personal data, the processing falls under the GDPR: the legal basis for processing must be established, data minimisation principles apply, and the right to erasure must be honoured — which in a vector index means being able to delete the vectors derived from a specific piece of data. If the AI system falls within a regulated category, the EU Artificial Intelligence Act also applies, with obligations around transparency and data governance. ISO/IEC 42001 provides an AI management system framework to structure that governance.
Common mistakes when working with embeddings
- Mixing embeddings from different models. Vectors from two different models are not comparable; switching models requires reindexing the entire corpus.
- Poor chunking. Chunks that are too large dilute meaning; chunks that are too small lose context. Chunk size and overlap are critical parameters.
- Relying solely on vector search. For proper nouns, codes, or exact figures, lexical search still outperforms; the robust approach is hybrid search (semantic + keyword).
- Ignoring the cost of dimensionality. 3,072-dimensional vectors multiply storage and latency compared to 384-dimensional ones; choose based on the use case, not by defaulting to the largest available.
- Forgetting the right to erasure. Indexing personal data without a mechanism to delete the corresponding vectors complicates GDPR compliance.
Frequently asked questions
What is the difference between semantic search and keyword search? Keyword search compares literal words; semantic search compares meaning through vectors, so it finds relevant results even when they share no words with the query.
Do I need a dedicated vector database? Not always. For moderate volumes, extensions such as pgvector on PostgreSQL are sufficient. Specialised databases (Milvus, Qdrant) offer advantages at scale and for advanced filtering.
Does RAG eliminate model hallucinations? It significantly reduces them by grounding responses in retrieved documents and allowing sources to be cited, but it does not eliminate them entirely: the model can still misinterpret the retrieved context.
Which similarity metric should I choose? For text, cosine similarity is the default. When vectors are normalised, the dot product is equivalent and more efficient. Euclidean distance is reserved for cases where magnitude carries information.
Conclusion: embeddings turn meaning into operable geometry
The power of vector embeddings does not lie in the model that generates them, but in the idea that meaning can be measured as distance. That translation is what allows a question phrased with words that appear in no document to still retrieve the correct answer, a corpus of millions of texts to be grouped by topic without prior labels, and a language model to respond about documents it never saw during training. The real value of an embeddings-based project is decided in the engineering of the details: the right model for the language and domain, a chunking strategy that preserves context, an ANN index that balances latency and precision, and a hybrid search that combines semantic retrieval with keyword matching so that proper nouns and exact figures are never missed. All of this must be underpinned by data governance that respects the GDPR and the AI Act from the design stage. At Summum Artificial Intelligence, we design semantic search and RAG systems end to end, choosing every component according to the real use case rather than the trend of the moment.