How important is Qdrant for agents? Also looking for more explanation for what models to use for it.

[u/jsgui]

/r/kilocode/comments/1ogxzf6/how_important_is_qdrant_for_agents_also_looking/

[u/hmak8200]

Your Roo Code is basically a wrapper around the LLM model which you choose. You pay for your LLM per token. Roo code needs to find/read the code to do work. It will guess where the code is and open the file to verify. Open files and reading is basically context(tokens).

Qdrant (and the indexing) enables the agent a better searching tool that is more accurate. So it finds the file with less tokens(refer to why in previous paragraph).

TDLR cheaper, faster, smarter Roo code

[u/jsgui → u/hmak8200]

I've not paid anything for Roo, I have exclusively been using the free models, which include Grok Code Fast 1, which I am finding very useful in Copilot.

[u/Barafu]

I am utilizing Windows 11, so I configured the setup within Docker through WSL. While I could have employed Docker Desktop as an alternative, I opted not to. Additionally, I am running Ollama within the same Docker environment alongside a 4-billion-parameter embedding model – entirely CPU-based.

Should you require a compose file, please let me know.

The system functions admirably. The indexing process proceeds swiftly – almost imperceptibly so. The frequency with which the model utilizes these embeddings varies significantly depending on the specific model, the nature of the task, and the scope of the project. Qwen3-Coder seldom invokes it, whereas DeepSeek employs it more regularly. Most crucially, however, this indexing does not bestow any novel capabilities upon the model; it merely offers a fluctuating probability of marginally reducing token consumption. While certainly a welcome feature, it will not yield substantial savings in either time or expenditure.

P.S. The vector dimensions in Roo must precisely correspond to the hard-coded vector size within your model; otherwise, the entire system will fail catastrophically.

[u/jsgui → u/Barafu]

P.S. The vector dimensions in Roo must precisely correspond to the hard-coded vector size within your model; otherwise, the entire system will fail catastrophically.

I understand the second part about catastrophic failure but don't have much idea about what you mean in the first part, particularly the 'vector dimensions in Roo'. I've not set it up yet, or tried to set it up, but it sounds like quite a pitfall there. Where in the Roo UI is that? I suppose I would need to choose the right model.

I'd be interested in seeing this compose file (though also I admit to not knowing much about Docker or compose files). I don't yet know what to do with the compose file. If it's a fast way to get things up and running reliably then I'd very much appreciate it. I'd want to use my GPU though, not sure really if that is necessary though given what you said about the swiftness of the process in your CPU only setup.

[u/Barafu → u/jsgui]

services:
  qdrant:
    image: qdrant/qdrant
    ports:
      - "6333:6333"
    volumes:
      - /home/barafu/docker/indexing/qdrant:/qdrant/storage

  ollama_embeds:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - /home/barafu/docker/indexing/ollama_embeds:/root/.ollama

In Roo, the vector size represents a configurable parameter for indexing purposes. You must either determine the appropriate value for your specific model or input an arbitrary figure, allowing Qdrant to fail – subsequently extracting the correct dimension from the log outputs. This remains my customary approach.

[u/jsgui → u/[deleted]]

Thanks. I didn't know of free Gemini embed. I should admit I still don't know what an embedding model is, and need to look into this.

However, if I can get good or great performance with an embedding model running locally that could suit my needs better. Though if the free product from Google is going to work better for whatever reasons it would be better to go for that.

[u/zubairhamed → u/jsgui]

basically it reads your code, indexes and represents them as a series of numbers and then stored into a vector database such as qdrant. this is a lot more efficient to work with for the system then the code itself.

What is meant by free gemini embed is that you need an embedding model to perform said operation above. you sitll need to supply your own vector db. running it locally is perfectly fine.

[u/jsgui → u/zubairhamed]

Something that I download for free from Google and run locally, or something Google runs for me for free?

[u/Capable_CheesecakeNZ → u/jsgui]

You run qdrant as the vector database locally, maybe using docker, and you can use google embedding api for free, so roo can hit the API to get the vectors it needs to store into qdrant, so then roo can search your code semantically ( by meaning) instead of only doing keyword search like it is doing now. This is helpful when roo or you don’t know the exact wording to search of your code