Keyword searches fail when users and documents use different terms to describe the same concept. Searching for "fuel-efficient cars" will miss articles that mention "cars with low fuel consumption." Embedding was developed to solve this problem.
Core Idea
A model that transforms a text passage into a vector with several hundred to several thousand dimensions. The key point is that two passages with similar meanings will result in two vectors that are close to each other in that space, even if they do not share a single word.
A common way to compare the closeness of two vectors is to use the cosine: a value of 1 indicates they are in the same direction, and a value of 0 indicates they are unrelated.
The Process of a Typical RAG System
- Split the document into appropriately sized segments, typically a few hundred words, with some overlap
- Compute the embedding for each segment and store it in a vector database
- When a question is received, compute the embedding of the question and find the closest segments
- Provide those segments along with the question to the language model for a response
Common Problems in Real-World Use
Cutting at the wrong place. Cutting in the middle of a sentence or separating a table from its heading causes the paragraph to lose context, making the embedding meaningless. You should cut along natural boundaries, such as section headings.
Rely solely on semantics. For error codes, proper nouns, and product codes, keyword searches are still more accurate. Combining both and re-ranking the results yields significantly better results than using either method alone.
Including too many passages. Cramming twenty passages into the context often makes the answer worse, because the correct information gets buried amid the noise.
The quality of RAG depends more on the retrieval stage than on the text generation stage. If the wrong passages are retrieved, even the best model won’t be able to help.
Thảo luận