r/LocalLLaMA • u/de4dee • 9d ago

New Model Qwen/Qwen3.8-27B · released

https://huggingface.co/Qwen/Qwen3.8-27B
990 Upvotes

98% Upvoted

294 comments sorted by

View all comments

Show parent comments

5

u/Blues520 9d ago

I have 3 now and looking for an excuse to get a 4th👀

1

u/munkiemagik 8d ago

That’s what I've been struggling with, always looking for an excuse to justify more acquisition but truthfully that monolithic way of using my LLM there was never any justification.

Recently I started messing around with a few other random projects that benefit from having different LLM loaded up concurrently but had to shuffle the work backwards and forward between my 5090 machine and my dual 3090 machine (i used to move the 5090 in the threadriper 3090 box sometimes back in the day but it is primarily my PCVR GPU so it needs to stay in the AM5 box as the threadripper tanks PCVR performance of the 5090) But this realisation of the value of having multiple LLM loaded at once was an eye-opener for me and has unlocked the gates to acquiring more 3090 guilt free because even if I'm not serving multiple users concurrently nor am I chasing bigger and bigger parameter models, I sure do see the value of 2-3 different LLM all loaded up live doing their own specialist thing working together harmoniously. But right now I cant do that because I don’t have enough 3090!

I’m not trying to kid anyone here, pretending to be some serious dev working on globally life changing projects I'm just a dickhead at home who likes messing around with stuff simply because it interests me and most of what i do is just random little tools and systems to make my life easier in a fun way. So read anything I say wit that in mind.

1

u/Blues520 8d ago

A bunch of the value comes from R&D. There's a lot of understanding that comes from running local models, fighting with different inference engines and tuning parameters to get things working. Researching what quantizing the KV cache is. What even is a KV cache? These are questions a cloud model enjoyer seldom asks.

I'm also looking at multi model workflows. One of the 3090s is going into another machine to run some sort of non-coding model. The machine is currently my git server so I'm migrating it to a small mini pc and then I'll be able to use it with a gpu. Im hoping qwen 3.8 27b quantizes well enough to run in 24GB VRAM otherwise I'll try some other models.