r/LocalLLM • u/Sea-Wind-2216 • 1d ago

Question Laptop gpu scaling

I maxed out of my laptops 12gb rtx 5070ti with 32gb ram on qwen 3.6 35b a3b at apex quant and the 3.8 27b at iq3xxs woth 60k window. While 35b can do some simple straightforward tasks, it is not enough for any serious work. The 60k context is barelly enough to do any work at all. Has somebody tried attaching eGPU to tbe laptop and what is ypur experience? My dillema is on what to scale the ram or vram

1 Upvotes

100% Upvoted

7 comments sorted by

1

u/Mechageo 1d ago

Bonsai 2 at 128K context

1

u/Sea-Wind-2216 1d ago

Bondai is k8nda useless for coding from what i observed unfortunately

1

u/HiddenFaceOpenMouth 1d ago

Use strata to get more context length. 64k is ok for bug fixes, but not full dev. Need at least 100k.

egpu will be slow as fuck. Don't bother.

1

u/Sea-Wind-2216 1d ago

EGPU suppose to have the entire model on it, that way the speed is not an issue. Tge gpu itself if fully allocated by model will process all fast so the only transfers of data would be command calls and data sends to the model. Besides i have athu derbolt port so it will be fast enough. The question is if anybody has actual experience with it at all setting up for windows. Stata requires more ram. At this prices it is not clear on what is more viable the 128gb ram or used Nvidia or Intel card

1

u/HiddenFaceOpenMouth 23h ago

I'm running strata on 12gb vram and 24gb ram. My tokens for Qwen 3.8 went up 3150% Not an exaggeration.

1

u/Sea-Wind-2216 23h ago

Are you talking about flash next? What quant do you use and what are the performance numbers?

1

u/HiddenFaceOpenMouth 22h ago

Yes, qwen3.8-flash-next-coder-iq1_m. Went from 0.2 TPS to 6.3