TOPIC AI
RAG grounding LLM answers in your own data
IN 10 SECONDS
Retrieval-Augmented Generation (RAG) adds a retrieval step before the LLM generates an answer. You embed a user's question into a vector, search a database of document chunks for the most relevant content, inject those chunks into the prompt as context, and then the LLM answers based on that retrieved information — not just its training data.
GOTCHA Garbage in, garbage out. If your documents are poorly chunked, outdated, or irrelevant, the retrieved context will be bad and the LLM will hallucinate around it. Invest in chunking strategy, embedding model quality, and document freshness.
HOW A RAG QUERY FLOWS
01 User asks a question: 'What's our incident response SLA?'
02 Embedding model converts the question into a vector (numeric embedding) that captures its semantic meaning.
03 Vector database finds the top-k document chunks whose embeddings are nearest to the question's embedding (cosine similarity).
04 Context builder stuffs the retrieved chunks into the LLM prompt as context, above the user's question.
05 LLM generates an answer using only the provided context — it doesn't guess from its training data.
POKE IT YOURSELF
curl -X POST -d '{"input":"What is RAG?"}' https://api.openai.com/v1/embeddings — generate an embedding for a query
pgvector 'SELECT * FROM docs ORDER BY embedding <=> $1 LIMIT 5' — query for nearest vectors in pgvector
Drill this topic →
~80 sec read