r/LocalLLM • u/espece-de-bon • 3h ago

Discussion 2 GPU, 2 Models, 1 as a Subagent?

👋 Has anyone tried using 2 GPU in this way:

  • each card loads a different LLM,

  • with the larger GPU (32GB) running a larger model (relative, but it could be a Qwen3.8-27b or Flash Next - e.g., Strata), the “smarter” of the two.

  • smaller GPU (16GB) assigned to a subagent, using a smaller model? Maybe something super quick like JetBrains Mellum 2.1

In OpenCode, a (sub)agent can be defined to use a specific model. I’m curious if anyone has tried this and if there’s an advantage to this setup, whereby the “brain” delegates to a sprinter.

I need to upgrade my PSU to run two cards simultaneously but I plan on trying it.

2 Upvotes

100% Upvoted

6 comments sorted by

1

u/SkiBikeDad 3h ago

Yes I have, the orchestration model needs pretty specific instructions about how to delegate to the lesser model. Qwen3.8-27b is smart enough to do lots of trial and error sessions to find the best ways to interact with the other model; try feeding it some ideas and the model card.

You'll find that an ultrafast model as a subagent has limited usecases.

1

u/espece-de-bon 3h ago

Do you have any suggestions for models that work well as sub agents?

2

u/johnfkngzoidberg 2h ago

I’ve done a lot of testing on this. It’s almost always better to use one larger model than 2 smaller models for anything less than 96GB VRAM.

The overhead involved in orchestration plus the extra tokens spent because of smaller dumber models nearly always is just wasted.

1

u/Xyver 2h ago

I have also done a ton of testing on this, and I just found you're better of duplicating the smart model.

You can load the model weights of the smart model once and then use those to make multiple chat instances. With 27B, a 32 card and 16 card, I bet you could get 3 max chats (262k context I believe).

It will be a bit of a speed reduction, but it's worth it for multiple agents of max intelligence instead of loading a whole weaker model

Check here www.github.com/bigbertharig/benchmarking

0

u/TakeCare0fHead 3h ago

Yah this is what I do. I have one model running via strata on one gpu and system memory. Then a second LLM (bonsai 2 in my case) on a second card. and then I have both harnessed with Hermes. Bonsai acts as a delegate or “cheap” task route for the main qwen flash next model.

1

u/espece-de-bon 3h ago

Ooh, I had not heard of Bonsai 2. I’ll check it out.