Deep Dive · RAG Pipeline | Strands Agent with Amazon Bedrock

📄 Why RAG Matters

A foundation model is trained on public data up to a cutoff date. It has never seen your onboarding policy, your fee schedule, or yesterday's dispute procedure update. Ask it "what's the representment window for a goods-not-received chargeback?" and it will answer confidently — and probably wrongly.

💡You're already doing manual RAG: every time you paste a policy into a chat and then ask a question, you're retrieving context by hand. RAG just automates that across thousands of documents.

The knowledge gap — and the fix

🧠

Models know public data only

They can't see your onboarding policy (ACP-ONB-001), fee schedule, or dispute procedure — so on internal questions they guess.

📎

Retrieve, then generate

Instead of retraining, RAG finds the relevant clauses and injects them into the prompt. The model reads them and answers — grounded and cited.

🔢

Powered by embeddings

Text becomes vectors, so meaning can be compared. "Goods not received" and "item never arrived" sit close together even with different words.

🔄

Always current

Update the document and the next answer reflects it — no retraining, and the model can cite exactly where the answer came from.

Where RAG sits in the stack

RAG vs fine-tuning in one line: fine-tuning changes how the model writes; RAG changes what it can see. For payments document Q&A — policies, fees, disputes — RAG wins almost every time, because your documents change and you need citations.

🔧 The Pipeline — six steps

Every RAG system follows the same flow. Here's what happens when a merchant-ops officer asks: "What's the representment window for a goods-not-received chargeback?" Click any step to explore it.

Step by step

Step	What happens	Payments example
❓ Query	Someone asks in natural language	"Representment window for a goods-not-received chargeback?"
🔢 Embed	The question becomes a vector	`[0.42, 0.78, -0.11, ...]`
🔍 Search	Find chunks with the closest vectors	Searches all chunked policy, fee, and dispute docs
📄 Retrieve	Top matches returned with scores	ACP-DIS-014 §3 (0.94), §6 (0.88), ACP-ONB-001 §5 (0.71)
🧩 Build prompt	Instructions + retrieved chunks + question	"Using only this context: [§3][§6]… answer and cite the clause."
✨ Generate	Model answers, grounded and cited	"7 calendar days (ACP-DIS-014 §3)." No hallucinated number.

Why not just paste the whole document? Large docs blow the context window, cost more per call, and dilute the model's focus. RAG sends only the few most relevant clauses — cheaper, faster, and more accurate.

🎬 Interactive RAG Demo

Walk a real payments query through the pipeline. Pick a scenario, then step through — or press play — to watch the query get embedded, matched against document chunks, and turned into a grounded, cited answer.

This is an illustrative walkthrough on synthetic documents — the vectors, scores, and answer are hand-authored to show the mechanics, not live inference.

Step 1 / 6

🧭 Vector space · 3D projection

Query Chunk Retrieved

🖱 drag to rotate
⚲ scroll to zoom

3D view unavailable offline — the console, chunk scores, and pipeline still show how retrieval works.

📚 Knowledge base

Watch the retrieve step: the chunks that light up are the ones whose vectors sit closest to the query — that's semantic search. The model only ever sees those few clauses, which is why it can cite them precisely.

✂️ Chunking — the quiet make-or-break

Before anything can be retrieved, your documents are split into chunks and embedded. How you chunk decides what can be found. Chunk too big and answers get diluted with irrelevant text; too small and you lose the context that makes a clause meaningful.

The trade-off

🧱

Too large

A whole policy in one chunk retrieves lots of noise around the answer — higher cost, less focus, weaker citations.

🔬

Too small

A single sentence loses its context — "7 days" without the clause it belongs to isn't a usable answer.

🎯

Just right

One coherent idea per chunk — a clause or sub-section — with a little overlap so context isn't cut mid-thought.

🏷️

Keep metadata

Tag each chunk with its document ID and section (ACP-DIS-014 §3) so the answer can cite precisely and retrieval can be scoped.

Good practice for payments documents

Split on structureChunk on natural boundaries — clauses, sections, table rows — not a blind character count.
Add overlapA small overlap between chunks keeps a clause from being severed from its condition.
Attach citationsStore the document ID and section with each chunk so answers are traceable and auditable.
Scope by accessTag chunks by tenant/merchant so retrieval can't cross boundaries (see Module 05, LLM08).
Refresh on changeRe-chunk and re-embed when a policy, fee, or procedure is updated — the index must track the source.

Rule of thumb: one clause or one idea per chunk, tagged with its source. If a human would quote that block to answer the question, it's a good chunk.

🏗️ Building Your RAG on AWS

You don't have to assemble the pieces by hand. Amazon Bedrock Knowledge Bases manages ingestion, chunking, embedding, the vector store, and retrieval — so you connect your documents and query them.

What a managed RAG gives you

📥

Ingestion & chunking

Point it at your documents (e.g. in Amazon S3); it chunks and embeds them for you.

🗂️

Managed vector store

A vector database is provisioned and kept in sync as documents change — no infrastructure to run.

🔍

Retrieve + generate

One API retrieves the relevant chunks and generates a grounded answer with source references.

🛡️

Guardrails built in

Attach Bedrock Guardrails and contextual-grounding checks so answers stay safe and grounded (Modules 04–05).

RAG vs the alternatives

Approach	How it works	Best when
Paste full document	Copy the whole doc into the prompt	One short document, one-off question
RAG ✓	Auto-retrieve relevant chunks, inject, generate with citations	Many documents, changing content, need to cite — most payments Q&A
Fine-tuning	Retrain the model on your data	A fixed style/format need — not for facts that change or must be cited

Best practices recap

Ground & cite"Use only the retrieved context; if it's not there, say so" — and show the source.
Chunk with careOne idea per chunk, tagged with its document and section.
Scope retrievalFilter by merchant/tenant and access rights before generating.
Check groundednessUse contextual-grounding checks to block ungrounded answers.
Keep it freshRe-index when source documents change so answers stay current.

↩ Module 06 · Implementing Projects 🛠️ Module 07 · Developing Solutions → ↩ Course Hub

Reminder: RAG is the single most useful pattern for a payments GenAI pilot — it turns a general model into an assistant that answers from your onboarding policy, fee schedule, and dispute procedure, with citations a reviewer can check.