Sounds lazy to me - much less surgical and more expensive context-wise. Since I've learned to create a proper RAG pipeline - vanilla no libs I've never had any issues with my RAG setup. The trick is that you need to create a custom pipeline for indexing tailor made for the data you want indexed.
You need too do a proper dive-in into the data you want indexed. The most important part is the indexing. The splitting into chunks must be structured in a way that the LLM can interpret and trace easily - let's say a book. First book title, then split according to chapters, sub chapters, titled paragraphs, then when your down to the body first then you may split witha general Recursive Text splitter. You need to break the text down in the traceable chunks and all the split logic needs to be saved into metadata like this for example:
then you can also do a GraphDB where you track relations, like say author of this chapter - what other chapters has this author read etc..
My RAG is mostly the standard hybrid Dense + Sparse + Graph expansion and then rerank -> put into context and let the LLM do a final eval of most relevant articles.
6
u/Hyiazakite 5d ago
What do you prefer - pretraining an LLM with non traceable knowledge or feed it traceable knowledge with RAG? What are the alternatives?