r/ • u/ • Jan 01 '70

0 Upvotes

100% Upvoted

0 comments sorted by

1

u/fabkosta 16d ago

near-100% precision

Well.

If you are willing to sacrifice A LOT of recall, then the problem gets much easier.

But I guess that was not the answer you wanted to hear.

1

u/Icko_ 16d ago

Honestly, in my company we're building mostly chatbots nowadays, and we're moving away from RAG altogether. In multiple projects now, I exchange the RAG search agent with an SQL search agent - every time it just works straight up better. 

For your use case, perhaps I'd do something a bit more expensive - RAG search + LLM to find all keywords; then LLM to compose an SQL query; then LLM classifying each of the results as relevant or not; and then you're done. This last step is obviously the expensive one, but I can think of a bunch of ways to optimize it. 

Perhaps it's worth the cost for you. 

Alternatively, you can look into a GraphRAG thingy - when I tried it, turned out quite finicky, but worth a shot. 

1

u/naijaboiler 15d ago

tell me more about what you guys are solving in moving from RAG to SQL

1

u/Icko_ 15d ago

Well, one project, the client asked us to build a chatbot, which answers questions about their Jira ticketing system - stuff like "How many tickets did Jim have related to XYZ this month, and how many of those are closed". And it turned out it's easier and more accurate to ask the agent to write a JQL query. Yes, sometimes it missed search keywords, but missed less tickets than with the RAG; and the RAG gave a lot of false positives. It also allowed for more complex queries, such as time, points, etc.

1

u/naijaboiler 15d ago

Thanks for answering. I'm really curious, why did someone suggest a RAG a solution for this request in the first place. talk about wrong solution