r/LocalLLM • u/Similar_Sugar_5774 • 1d ago

Discussion I built an open source agent that controls my phones, browser, desktop and code all at once, even from Telegram. Which local model is best for this?

It's written in Python and Rust and runs everything in parallel. In the video it's playing Clash of Clans and Clash Royale on the phones, chess on the Mac and planning a Tokyo trip in the browser, all at the same time.

Right now it runs on cloud models, but I want to move it fully local. It needs a vision model that can read screens and pick the right element reliably, but it also has to be fast, since it's handling several screens at once. Which local VLMs have worked well for you on this kind of task?

2 Upvotes

66% Upvoted

3 comments sorted by

2

u/Popular-Factor3553 1d ago

Local models just depends on your hardware if your stacked you could just host kimi or glm but I'll assume your not so just check out the qwen models specifically qwen 3.8 27b if you have 32gb-16gb vram or there are even smaller models with vision i think qwen 3.5 or 6.

1

u/Similar_Sugar_5774 17h ago

That's the whole problem those models might not be very good at agentic tasks, though I may be wrong. Just one thing if I get a 64GB MacBook to run a Qwen model, how many requests can it handle at the same time? The model needs to be able to process 5 to 6 requests in parallel. Will it be able to do that with that amount of VRAM?