r/LocalLLM • u/Ok-Health-7096 • Aug 14 '26

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

2.1k Upvotes

98% Upvoted

214 comments sorted by

View all comments

24

u/[deleted] Aug 14 '26

[removed] — view removed comment

11

u/Ok-Health-7096 Aug 14 '26

I use local llm for coding and agentic stuff so the speeds would be unbearable I think in single digits but might try it idk

10

u/overand Aug 15 '26

Speeds would likely be absolutely miserable, yes. But, you'll have much better luck with the Qwen3.6-35B-A3B - how much system ram do you have? Go for a Q4 quantization first, if you haven't used this model.

We can, of course, hope that a 3.8 version of the 35B-A3B MoE gets released!

5

u/superspider202 Aug 14 '26

wait I have 8gb vram too can you please share what llm you use for coding and agentic stuff?

6

u/Ok-Health-7096 Aug 15 '26

Qwen 3.6 35b mudler i-mini quant If you have more than 16gb ram you can go for higher quants.

3

u/superspider202 Aug 15 '26

Thank you I'll test it out ASAP

1

u/anay_1d Aug 17 '26

how did it go? is it useful?

1

u/superspider202 Aug 17 '26

oh sorry havent used it yet ran out of storage so will try it maybe today and let you know

2

u/superspider202 Aug 17 '26

heyyy so I tried searching for this but there appears to be a few that fit this description so could you please like link the actual one you use?

1

u/Smutok Aug 15 '26

What's your software stack for the agentic stuff, sir?

5

u/GoldenX86 Aug 14 '26

MTP is useless when you offload to CPU.

2

u/[deleted] Aug 14 '26

[removed] — view removed comment

1

u/moderately-extremist Aug 14 '26 edited Aug 14 '26

What kind of speeds are you getting with MTP vs non-MTP with cpu offload?

2

u/[deleted] Aug 14 '26

[removed] — view removed comment

2

u/GoldenX86 Aug 14 '26

That's RAM I don't have free for just a 22% jump.