How RAG, chunking, embeddings, indexing, and retrieval fit together.
What is RAG
Retrieval-Augmented Generation (RAG) combines a retrieval system with a large language model. When a user asks a question, the platform first retrieves relevant passages from a Knowledge Base, then sends those passages alongside the question to the LLM to generate a grounded answer.
| Dimension | LLM alone | RAG |
|---|---|---|
| Knowledge freshness | Frozen at training time | Real time — follows your data |
| Private data | Not available | Connects to internal data |
| Source attribution | None | Returns the citations and sources |
| Hallucination control | Weak | Strong — answers cite real passages |
Processing pipeline
| Stage | What it does |
|---|---|
| Parsing | Identifies headings, paragraphs, tables, images, and formulas and stores them as structured content. |
| Chunking | Splits documents into retrievable pieces, typically 256–4096 tokens. |
| Embedding | Converts text into vector representations for semantic matching. |
| Index | Stores Chunks and vectors for fast search. |
| Retrieval | Returns the top-K most relevant Chunks for a query. |
| Rerank | Re-orders the initial results to improve relevance. |
| Generation | The LLM produces the final answer based on the retrieved Chunks. |
Knowledge Base types
| Type | Data | Retrieval method | Typical use cases |
|---|---|---|---|
| Unstructured Documents | PDF, Word, Markdown, HTML | Hybrid vector + keyword | Product manuals, contract templates, internal wikis |
| Structured data | CSV, Excel, RDS tables | Natural-language-to-SQL | Order data, inventory reports |
| Image library | Product photos, design files, posters | Image-to-image search, visual Q&A | Image search, visual Q&A |
| Audio / video library | Meeting recordings, training courses | Speech-to-text retrieval | Video content search |
Glossary
| Term | Definition |
|---|---|
| Knowledge Base | A logical collection of documents that share a chunking, embedding, and index configuration. |
| Document | The raw content uploaded or synced into a Knowledge Base — a file, web page, or database record. |
| Chunk | The smallest unit of retrieval; a document is split into Chunks. |
| Top-K | The number of Chunks returned by retrieval — typically 3 to 10. |
| Relevance score | The semantic similarity between a Chunk and the query, on a 0–1 scale. |
| Data Connector | An ingestion channel for an external source — OSS, MySQL, Yuque, and so on. |
| Knowledge Search service | A configuration unit for Multi-index Retrieval, with its own tunable parameters. |
| Knowledge Q&A service | Adds LLM generation on top of retrieval and returns a natural-language answer with citations. |
| Workspace | A resource isolation boundary — one account can have multiple workspaces. |
Retrieval modes
| Mode | How it works | Best for |
|---|---|---|
| Fast mode | Single-shot retrieval with the lowest latency. | Simple factual questions. |
| Agentic mode | Model-driven multi-turn planning that rewrites the query as it goes. | Complex or multi-step questions. |