Embeddings turn text into vectors — points in space — so that similar ideas sit close together. It's how a model knows "goods not received" and "item never arrived" mean nearly the same thing, and it's the engine behind semantic search and RAG.
An embedding is a list of numbers — a vector — that captures the meaning of a piece of text. Words and phrases with similar meaning get similar vectors, so they land near each other in space. That geometry is what lets a computer compare meaning, not just match keywords.
An embeddings model maps text to a vector of hundreds of numbers — coordinates in a high-dimensional space.
Vectors close together mean similar things; far apart means different. Different words, same meaning still land close.
Search by meaning, not exact words — "item never arrived" finds a "goods not received" clause.
RAG embeds your documents and the query, then retrieves the chunks whose vectors are nearest.
This is a 3-D projection of a high-dimensional embedding space, grounded in payments vocabulary. Terms that appear in similar contexts cluster together. Drag to rotate, scroll to zoom, and click any word to light up its nearest neighbours — exactly what a semantic search (and RAG) would retrieve.
Pick two words to compute their cosine similarity — the standard measure of how "close" two embeddings are. The pair is also drawn as a line in the 3D space above.
Once text is a vector, "how similar are these two meanings?" becomes a geometry question. The usual measure is cosine similarity — the angle between two vectors, scored from -1 (opposite) through 0 (unrelated) to 1 (identical meaning).
Compares the direction of two vectors. Same direction = same meaning (≈1); perpendicular = unrelated (≈0).
To find relevant text, score the query vector against all stored vectors and take the highest.
A vector database indexes millions of vectors so nearest-neighbour search stays fast.
| Metric | What it measures | Range | Best for |
|---|---|---|---|
| Cosine similarity ⭐ | Angle between two vectors (direction) | -1 to 1 | Comparing meaning regardless of vector length |
| Euclidean distance | Straight-line distance between points | 0 to ∞ | When absolute magnitude matters |
| Dot product | Direction and magnitude combined | -∞ to ∞ | Fast approximate similarity at scale |
| Pair | Cosine similarity | Meaning |
|---|---|---|
"goods not received" vs "item never arrived" | ~0.93 | Nearly the same — different words, same intent |
"chargeback" vs "representment" | ~0.88 | Closely related in the disputes domain |
"MDR" vs "PayNow fee" | ~0.82 | Same topic (fees) |
"chargeback" vs "MDR" | ~0.35 | Both payments, but different topics |
"card testing" vs "settlement timing" | ~0.20 | Largely unrelated |
Embeddings are the bridge between raw text and everything a model does with meaning. Here's where they sit in a grounded (RAG) payments assistant — and everything else they quietly power.
Both your documents (once, ahead of time) and the user's question (at query time) are embedded into the same space — retrieval finds the document chunks whose vectors are nearest the question. That's RAG.
| Piece | Service | Role |
|---|---|---|
| Embeddings model | Amazon Bedrock (Titan Text Embeddings) | Turns text into vectors |
| Vector store | Amazon OpenSearch / Aurora pgvector (or managed by Knowledge Bases) | Stores and searches vectors at scale |
| Managed RAG | Amazon Bedrock Knowledge Bases | Handles embedding, storage, and retrieval for you |
Before dense embeddings, the simplest way to turn words into numbers was one-hot encoding: give every word in the vocabulary its own slot, and represent a word as a giant vector that's all zeros except a single 1 in that word's slot.
| Word | One-hot vector (illustrative, tiny vocab) |
|---|---|
| chargeback | [1, 0, 0, 0, 0, …] |
| dispute | [0, 1, 0, 0, 0, …] |
| settlement | [0, 0, 1, 0, 0, …] |
| PayNow | [0, 0, 0, 1, 0, …] |
One slot per word means vectors as long as the whole vocabulary — tens of thousands of dimensions, almost all zeros. Wasteful and slow.
Every word is equally far from every other. "chargeback" and "dispute" are exactly as distant as "chargeback" and "PayNow" — the encoding knows nothing about meaning.
A new word (or a typo, or an internal code) has no slot at all — there's nowhere to put it.
A dense embedding represents each token as a much shorter vector — typically a few hundred numbers — where the values are learned so that similar meanings end up with similar vectors. Instead of one slot per word, meaning is spread across all the dimensions.
| Property | One-hot | Dense embedding ⭐ |
|---|---|---|
| Size | = vocabulary (tens of thousands) | A few hundred dimensions |
| Values | All 0 except one 1 | Learned decimals, e.g. [0.42, -0.11, 0.78, …] |
| Captures meaning? | No — every word equidistant | Yes — similar words sit close together |
| Handles new phrasing? | No | Yes — "item never arrived" lands near "goods not received" |
An embeddings model is trained on huge amounts of text; words that appear in similar contexts are nudged to similar vectors.
Each dimension captures some latent aspect of meaning. Related concepts point in similar directions in the space.
A few hundred numbers per token is cheap to store and quick to compare at scale — millions of chunks stay searchable.