RAG: إزاي تخلي الذكاء الاصطناعي يجاوب من بياناتكRAG: making an AI answer from your own data

RAG stands for Retrieval Augmented Generation, and it simply means: before the model answers, search your documents for the passages related to the question, put them in front of it in the same request, and say “answer from these”. The model is not trained on your data and does not memorise it. It gets a cheat sheet with every question. This is the most common way to make a chatbot answer about your product, your policies or your project’s docs.

The problem it solves

A language model has two problems with your data:

• It does not know it. Your refund policy, prices and project docs are not in its training.
• When it does not know, it invents. It writes a convincing answer from imagination.

The obvious fix is to paste all your documents into every question. That works for two pages. For a thousand, it becomes expensive and slow, and the model gets lost. RAG solves this: it brings the model the two pages it needs out of the thousand.

The idea in four steps

1. Preparation (once): split documents into small chunks, turn each into an embedding, and store them.
2. Retrieval (per question): turn the question into an embedding and fetch the closest chunks.
3. Assembly: put those chunks with the question in one request.
4. Answer: the model writes the reply from the chunks and says where each fact came from.

What an embedding is

An embedding is a long list of numbers that represents the meaning of a text. A dedicated model takes a sentence and returns, say, 1000 numbers. Sentences with similar meaning get numbers close to each other, even when the words differ completely.

Example: “how do I return the product?” and “refund policy” share no word, yet their embeddings are close. Keyword search misses that link, and semantic search finds it.

These numbers are stored in a database that can search by closeness, called a vector database. It can be a separate service, or an extension to a database you already have, such as pgvector for PostgreSQL.

Chunking: the step that decides quality

Most RAG problems come from here, not from the model.

• Chunk too small: a sentence without its context. “The period is 14 days” of what?
• Chunk too large: five topics inside, so its embedding represents none of them well.

What usually works:

• Split on meaning boundaries: a heading and its paragraphs, not every 500 characters.
• Keep a small overlap between consecutive chunks so a sentence in the middle is not cut.
• Add the document title and section to each chunk. That gives it back its context.
• Store the source with each chunk: file name and link, so the answer can point to it.

What the code looks like

This is the skeleton in JavaScript, without specific library names. embed, search and ask are functions you wrap around whichever service you use:

// once: prepare the documents
for (const doc of documents) {
  for (const chunk of split(doc)) {
    const vector = await embed(chunk.text);
    await store.insert({ vector, text: chunk.text, source: doc.url });
  }
}

// for every question
async function answer(question) {
  const qVector = await embed(question);
  const chunks = await store.search(qVector, { limit: 5 });

  const context = chunks
    .map((c, i) => `[${i + 1}] ${c.text}nSource: ${c.source}`)
    .join('nn');

  return ask(`
Answer the question from the passages below only.
If the answer is not in them, say "I cannot find this in the documents".
Write the passage number after each fact.

Passages:
${context}

Question: ${question}`);
}

The two most important lines in the request: “from the passages only” and “if it is not there, say so”. Without them the model fills in from itself when retrieval returns something unhelpful.

Common mistakes

• Evaluating by feel. Write 30 real questions with correct answers and run them after every change. Otherwise you cannot tell whether a change helped or hurt.
• Relying on semantic search alone. Product codes, names and numbers are found better by keyword search. Combining both is strongest.
• Conflicting documents. With two versions of one policy, the model may answer from the old one. Clean the source before indexing.
• Forgetting updates. The document changed and the index is stale. Build a way to re-index what changed.
• Permissions. If some documents are not for all users, filtering must happen in retrieval itself, not be left to the model.
• Documents containing instructions. Text in a document may try to steer the model. See prompt injection.

If retrieval brings the wrong passage, the best model in the world answers wrongly.

When you do not need RAG

• Your data is small. If all documents fit comfortably in the request, include them all. Simpler and more accurate.
• The question is about structured data. “How many orders this month?” is an SQL query, not a text search. Give the model a tool that queries the database.
• You have an agent that can search. In a code project, an agent that opens and searches files itself gets good results with no index. Details in AI agents: what they are and how they work.
• You want to change the model’s style, not its knowledge. That is a job for instructions and examples, not RAG.

Start with the simplest thing that works: your documents directly in the request. When you outgrow that, add retrieval. If you want to build something like this for your work and do not know where to begin, write to me or see the AI automation course.