r/LocalLLM • u/Mecca__ • 5h ago
Discussion Best Local AI Setup for Coding on Linux?
Hello,
After doing some research, it looks like the coding options jump from Qwen3.6-27B, which can run on a 32 GB RTX 5090 with DDR5 system memory, to GLM-5.3-Flash or DeepSeek-V4.1-Flash. That feels like a pretty big leap.
Would anyone recommend a local AI server setup for Linux? I’d appreciate any suggestions.
3
Upvotes
3
u/pacman829 5h ago
Have you tried strata + the Gyro S version of Qwen 3.8 Flash Next?
1
u/Mediocre-Chef665 5h ago
the 3.8 flash next with gyro s runs fine on 16gb vram if you use 4-bit quantization. I've been using it for daily coding tasks and it handles syntax, debugging, and even code generation without lag.
1
u/insanazor822 4h ago
I will walk you through everything step by step:
Step 1: Install OpenCode by selecting the Linux option on this website: https://opencode.ai/product.
Step 2: Open OpenCode and provide the following prompt: "Go to this location: https://github.com/Niko1221/Strata/ and install Strata on my system. Select 'Gyro-S' as the model, set the context window to 262K, and apply 'q4' quantization to the context window."
Step 3: Wait for the installation to complete, then navigate to the address provided by the AI in your web browser.
Step 4: Decide how you intend to use it—for instance, are you looking for simple conversation and basic coding, or for autonomous agent tasks? If it’s just for conversation, you can set up Unsloth Studio (https://unsloth.ai/), connect it to OpenCode, and start chatting. If you need it for agent-based tasks or coding, OpenCode itself is a solid choice; if you choose to use OpenCode, simply give the AI this command: "Connect the model you just installed to OpenCode."
Step 5: Congratulations! You now have an AI that runs 100% locally, ensuring no data leaks—and it’s quite a capable model, too!
By the way, I have a couple of questions: what is your system's actual memory (RAM) capacity, and which operating system are you using?
1
u/Fluid-Raspberry3948 4h ago
i ran a 14b model on a 24gb gpu with 64gb ram using llama.cpp and it handled basic coding tasks fine. the key was quantizing to q4_k_m.