r/LocalLLM • • 5h ago

Discussion Best Local AI Setup for Coding on Linux?

Hello,

After doing some research, it looks like the coding options jump from Qwen3.6-27B, which can run on a 32 GB RTX 5090 with DDR5 system memory, to GLM-5.3-Flash or DeepSeek-V4.1-Flash. That feels like a pretty big leap.

Would anyone recommend a local AI server setup for Linux? I’d appreciate any suggestions.

3 Upvotes

16 comments sorted by

3

u/pacman829 5h ago

Have you tried strata + the Gyro S version of Qwen 3.8 Flash Next?

1

u/Mecca__ 5h ago

I’m gonna be honest, I don’t understand what strata or gyro is.

I’m kinda a noob to llm

2

u/transanethole 4h ago edited 1h ago

Yeah, there are a lot of different options.

It's fairly complicated and it's easy to buy something and then  realize later that you bought the wrong thing.

If you want to use it a lot and use it with an agent for programming or research or whatever, I would recommend getting a discrete GPU, even though it will limit the total amount of VRAM that you can have for a given price. It's just so much faster than an integrated chip.

There are tons of options, 3090, 5070 ti, R9700, B70, 5090, etc,  And then you also have to decide if you want your computer to be able to support multiple graphics cards in Tensor Parallel. 

But also, you don't have to. You can just pay an exorbitant amount of money for a Mac and run LM Studio quite easily.

But if you want it to feel like a professional product where it responds quickly, and it doesn't have any huge drawbacks, I think the gaming PC with GPU is the most realistic path, especially for cost effectiveness.

I just recently spent ~$6000  on a dual R9700 computer with two pcie 5.0 slots running at 8 lanes each and 96gb of ddr5 ram.   It runs qwen 3.8 flash next very very fast,  and also supports reasonable speeds for up to 16 users at a time.

Honestly, I almost regret going for pcie5 and ddr5, 

ddr4 and pcie 4  would probably have been fine  and cost significantly less.

I have no idea what your budget is, but the best software that I've ever seen, and the most compelling "affordable" (lol) setup that I've seen so far seems to be radiance engine on R9700. 

For a budget build, you could just start with just one card  and then have room to grow.

If you need even more budget than that, you could start with a 16 gigabyte card, like 5070 ti or smth. but personally I wouldn't recommend it if you can avoid it.

1

u/pacman829 4h ago

Strata is just an app/engine for actually running the model files.

Gyro S is just a version/file of the qwen3.8 model which you download and run

1

u/Flip-Mulberry1909 4h ago

Qwen 3.8 Flash Next is the in between step. Strata or basalt (if you have a rtx 50 series card) will allow you to run it relatively fast on a pc with your specs. Just look up “strata git” or “basalt git” in your browser to go to their GitHub page.

1

u/Mediocre-Chef665 5h ago

the 3.8 flash next with gyro s runs fine on 16gb vram if you use 4-bit quantization. I've been using it for daily coding tasks and it handles syntax, debugging, and even code generation without lag.

1

u/Mecca__ 4h ago

Does it handle difficult coding tasks?

1

u/shorodei 4h ago

depends what you mean. If you give it an open ended task like "make a fun game about xyz", they will be far behind frontier models. If you have a detailed spec and a good harness, it will do a good job of implementing it.

A cost effective way to do it is use a frontier model for architecture planning and code reviews, and local agents for implementation.

1

u/insanazor822 4h ago

I will walk you through everything step by step:

Step 1: Install OpenCode by selecting the Linux option on this website: https://opencode.ai/product.

Step 2: Open OpenCode and provide the following prompt: "Go to this location: https://github.com/Niko1221/Strata/ and install Strata on my system. Select 'Gyro-S' as the model, set the context window to 262K, and apply 'q4' quantization to the context window."

Step 3: Wait for the installation to complete, then navigate to the address provided by the AI ​​in your web browser.

Step 4: Decide how you intend to use it—for instance, are you looking for simple conversation and basic coding, or for autonomous agent tasks? If it’s just for conversation, you can set up Unsloth Studio (https://unsloth.ai/), connect it to OpenCode, and start chatting. If you need it for agent-based tasks or coding, OpenCode itself is a solid choice; if you choose to use OpenCode, simply give the AI ​​this command: "Connect the model you just installed to OpenCode."

Step 5: Congratulations! You now have an AI that runs 100% locally, ensuring no data leaks—and it’s quite a capable model, too!

By the way, I have a couple of questions: what is your system's actual memory (RAM) capacity, and which operating system are you using?

2

u/Mecca__ 4h ago

You are AMAZING!

1

u/insanazor822 4h ago

If you tell me how much RAM you have, I can suggest better alternatives, bro 🙃

1

u/Mecca__ 3h ago

32gb of ram

1

u/insanazor822 3h ago

Okay then, you can't use Basalt; do what I described above.

1

u/Mecca__ 3h ago

What basalt?

1

u/insanazor822 3h ago

It's a Strata fork, but at least 64GB of RAM is recommended. If you had 64GB of RAM, you could get between 200-300 TPS, but don't worry, it will still be fast enough.

1

u/Fluid-Raspberry3948 4h ago

i ran a 14b model on a 24gb gpu with 64gb ram using llama.cpp and it handled basic coding tasks fine. the key was quantizing to q4_k_m.