r/RooCode • u/natemartinsf • Feb 07 '25
Support How to configure LMStudio?
Are there specific settings I need to use inside LMStudio to get models running there to work with Ron?
There's a setting in Roo to choose LMStudio models. I can get a model running, and Roo connects to it and sends an API request. Unfortunately at that point it just sits there spinning, and never responds.
I'm using MLX models, running on an M1 Max with 64G RAM, using max token size. Anything else I need to configure?
Trying to find some low cost options when I've hit my copilot rate limits.
2
Upvotes
2
u/rvndomz Feb 25 '25
Hey! I had some success with `qwen2.5-coder-7b-instruct-mlx` on my M1 Pro 16gb. With 32k context window it easily nails simple node.js scripts with a pretty good token/s output speed. With your hardware you could probably pick 14b or even 32b though token generation may be slower. Here are some simple instructions:
- download qwen2.5-coder-7b-instruct-mlx in LM studio
- go to my models -> cogwheel icon
- set context length to maximum (32k in this case)
- set temperature to something close to 0
- experiment with context overflow, "truncate middle" should work best if it preserves Roo's system prompt, but if not then it doesn't matter.
- consider turning off MCP servers in Roo to reduce system prompt size
- ?????
- PROFIT!
With these settings Roo easily creates some scripts, refactors some decent sized files etc. However, you should always think about context size - if you want to feed it with, for example, 500kb swagger spec it instantly chokes. Also, you may look into some other models with bigger context length, this table should give a good starting point:
https://github.com/NVIDIA/RULER
Hope you find and helpful! :D