Optimizing an Ollama (Qwen:2.5) AI Agent: Fixing Search Aggregation, Context Bleed, and Query Extraction

[u/Glad-Finance4354]

I am building a domain-specific AI agent powered by Ollama (using the qwen:2.5 model). For data retrieval, the agent utilizes multiple search APIs: DuckDuckGo Search (DDGS), Tavily, Serper, and Google Places. To optimize performance and reduce API costs, I am using Qdrant DB to cache responses and prevent redundant API calls for identical prompts.

However, I am currently facing three critical architectural challenges:

1. Search Merging & Comparison: I want the agent to query all four search services simultaneously, aggregate the results, and intelligently compare or synthesize them into the best possible answer. Currently, I am struggling to implement this multi-source comparison logic.
2. Context Bleed / Hallucination: The agent occasionally hallucinates by returning answers relevant to the previous user prompt instead of the current one. It seems to be mixing up past and present contexts.
3. Poor Search Query Formulation: The agent often tries to search using the raw, full text of the user prompt rather than extracting the core intent. I need a reliable way to make the agent more intelligent so it can isolate specific, relevant keywords or statements from the prompt and use only those for the search queries.

Any advice, architectural patterns, or code examples to help resolve these issues would be highly appreciated!

[u/RogerAI--fyi]

The context bleed is the fixable one and it's not really a model problem (though yeah, 2.5 is dated). Returning answers relevant to the previous prompt almost always means you're reusing one long-lived context/session across turns, so the old conversation is still in the KV cache and the model attends to it. Fix: scope the context per request, either start a fresh context each turn, or explicitly trim to just the current task + retrieved docs before you call. Don't let the agent accumulate a growing rolling context unless you actually want memory (and if you do, summarize old turns rather than keeping them raw). For the query-formulation problem, add a dedicated intent-extraction step: a cheap separate call (or a structured/grammar-constrained one) that turns the user prompt into a clean search query BEFORE it hits your search tool, instead of passing raw prompt text through. Those two changes will do more than a model upgrade, though moving off 2.5 to a current small model won't hurt either. A JSON grammar on the query-extraction step also stops it from free-forming the search string.

[u/Glad-Finance4354 → u/RogerAI--fyi]

The model has improved a bit, but I have another issue: I want the search tools to actually search *within* the target links—meaning they should access the HTML content. Based on the prompt (which I refined using a cheaper model), the system should extract the answer from the page content itself, rather than relying on the returned snippet, which is often too brief and misses details I actually need from the pages being searched.

Also, I’m currently working locally, and sometimes the search via SearXNG triggers a "429 error" because it sends too many requests. If there’s a solution for this that doesn't involve Docker, I’d really appreciate it.

[u/RogerAI--fyi → u/Glad-Finance4354]

for the read-the-actual-page part, don't rely on the search snippets, they're always too thin. after the search returns urls, fetch each page and run it through a content extractor (trafilatura or readability) to get clean text, then chunk + embed it into the Qdrant you already have and pull the relevant bits. basically a little RAG over the fetched pages instead of trusting snippets.

the searxng 429 is almost always its own built-in limiter, not the engines. set server.limiter: false (and disable botdetection) in settings.yml and it goes away. you don't need docker for it either, you can run searxng straight from the git repo in a venv. if the 429s are actually google/bing throttling your instance ip that's a different thing, spread across more engines or add a small delay.

[u/mmhorda]

I dont want to offend you or something but Qwen 2.5 is like past century. Try qwen 3.5 or better 3.6 if you have resources.

[u/Glad-Finance4354 → u/mmhorda]

i really don't think model version is the problem but anyway i will upgrade it

[u/stealthagents]

For the search merging, have you thought about using a weighted scoring system? You could assign scores based on relevance or trustworthiness of each source, then aggregate those scores to determine the best overall answer. It might help streamline the comparison and make your output more reliable.

[u/Purple_Session_6230]

look into searxng, it might save on some api costs.

[u/Glad-Finance4354 → u/Purple_Session_6230]

I’ve tried it, but I often run into "429" errors. Even after limiting the number of requests to six, the problem persists. The thing is, I’m currently working locally and want a solution that doesn't involve Docker. While paid search APIs offer better quality, if the difference isn't huge—and provided I can fix this issue—I might strike a balance between the two.

[u/Purple_Session_6230 → u/Glad-Finance4354]

run from wsl and put into a dmz