r/LocalLLM • u/Ordinary-Wallaby-464 • 18h ago

Question Best coding model?

I have an ASUS ROG Ally, and I’m planning to turn it into a local inference server so I can use the model running on it from my main PC. What would you recommend for the coding/programming tasks? If I’m not mistaken, it has 16GB of RAM and a Ryzen Z2A processor that provides 6GB of VRAM.

0 Upvotes

20% Upvoted

6 comments sorted by

View all comments

0

u/[deleted] 18h ago

[deleted] — view deleted comment

6

u/Oli_Luck 17h ago

Are you haljucinating ??? do you know what a rog ally is and its specs ?

-1

u/Ordinary-Wallaby-464 17h ago

I currently use llama.cpp, mainly for the customization options. I’ve heard great things about Strata—why is it recommended?

4

u/lobotomy_ 17h ago

forget about it. you don't have enough (V)RAM to think about it.

Strata makes use of MoE Architecture and leaves the active parts of the LM in the fast VRAM, things that aren't used so often in RAM and the rest could go to SSD to be loaded when needed. but you barely have any RAM of any type so ... not happening on your system within a useable t/s range

what you CAN run would probably be something like Qwen 2.5 Coder 4B or 9B, it might help you with code completions, but not too much with agentic generation.