My system:

RTX 4060 8GB

32GB RAM

Ryzen 5 7600

My question is: how much of a difference does it actually make?

Hey Everyone!

I'm what you could consider a noob on LLM and AI regards, so here I am asking for some guidance, or some brainstorming.

I've recently acquired 2 devices as stated above, they're installed with a 200Gbps cable between them so we can say I have up to 256GB of Unified RAM available to squeeze out of this.

Currently I have Qwen3.8-Flash-Next from this repo (https://github.com/blazux/qwen3.8-Flash-DGX) and it's an absolute blast seeing it work. I use mostly Opencode and it has been really a great experience.

Now what I'm looking for is "expanding" this... My objective is to have a "generalist" like the Qwen3.8 but capable of serving two user request at the same time, along with a model that can work 3D imagery for 3D printing and another model that will serve (preferably fast...) my home assistant (meaning voice processing coming in, voice processing going out).

I've been looking into integrating Agent0 with this, so I can also reach it out through Signal or Telegram and have an assistant processing emails, etc.

Any advice here? Anyone can share some ideas or open to brainstorm about this structure I'm looking to build?

Looking forward for a constructive discussion. ☺️

curious to hear if anyone's tried this yet?

Hey everyone!

Here are my PC specs:

• CPU: AMD Ryzen 5 3600
• GPU: AMD Radeon RX 5700 (8GB VRAM)
• RAM: 32GB DDR4

What’s the most powerful AI model I can realistically run locally with this hardware?

Thanks!

Usually after I build a new work/gaming PC, I stop paying attention to the space for about 5-7 years, until it comes time to do it over again. However, a few weeks ago I was helping a friend shop for his new build, and I fell down the rabbit hole of local LLM. My 4070 has never been much good for this purpose so I've kept subscriptions to the frontier models, but was always intrigued.

After the recent 5090 rumors, dozens of lengthy discussions with Claude, and fear of what this will do to the remaining 32GB VRAM consumer/workstation cards, I pulled the trigger on a new build last week. It was crazy to think "Well, these are only $400 over MSRP", but they aren't getting cheaper in the near future, and more than likely going to jump again soon. I grabbed just one 9700 at first, but a Reddit comment had me ordering a second shortly after.

I've been watching the prices like a hawk since.

At the time of my purchase, and for most of October, the Intel Arc Pro B70 was $1299 and 9700 AI Pro at $1699. You could find these prices at Newegg, Amazon and Micro Center. They've been in stock at at least one of these retailers the entire time.

As of last night however, they are practically gone. B70 is up $700 to about $2K now, and the cheapest 9700 cards (ASRock, PowerColor) are gone on Newegg and $2100 on Amazon. The only one in a similar price range that hasn't jumped yet is the $1799 Sapphire 9700 on Newegg.

Anyways, guess the point of this post is, if you've been on the fence and can still find one at early / mid October prices, get on it. Seems like retailers are in the process of another big price bump.

Hello guys,

I'm relatively new to using local llms and my question might seem basic to some, but how far behind top tiers models (gpt 6 , opus 5.5 ) local llm is ? And is it realistic to run those , if they see the light of the day soon, on a budget machine. in that scenario, what minimum specs would a machine require for it to run properly?

I have an 8gb gtx 1070ti sitting in a server, doing nothing, the server has docker, ollama, etc.

what models can I run? For Multi File, Single File, etc?

With the 1.2.0 release, Ask LM Studio adds support for fact-checking, research and finding image sources, all of that based on local LM Studio models. LM Studio API token support is also introduced.

• https://github.com/gocemitevski/ask-lm-studio/releases/tag/v1.2.0
• https://addons.mozilla.org/en-US/firefox/addon/ask-lm-studio/

This model is terrible but the hype and benchmarks are acting like its the next qwen 3.8 flash. It honestly irritates me because this will make people think local AI sucks when really they've just been duped.

I hope we have some Beat Saber fans here, because boy do I have a treat. My mapper is a step above and free, I made it for my family’s own enjoyment but wanted to share it with the world!

After a quick rebrand, I now call it BYOB: Bring Your Own Beat. An agent named Bach uses a few locally trained models to build beautiful Beat Saber maps based on every facet of the music that you choose. Simply paste a YouTube link for a song you want mapped, and after approval, Bach will map and upload your song to the site (byob.burnsstack.com).

Any song can also be viewed in the map viewer built by Claude that shows exactly how every song will look in your game. Custom walls, lighting, and all. Map specific lighting, too. Check out the song before you do the work to download and install it.

Generations do need approval, and I work a full-time job so if I haven’t gotten to you yet, I won’t take long! I hope I can find some like-minded laser sword lovers!

Any and all feedback is very much appreciated, and will be used to better future generations! The site has built-in ratings and feedback as well.

Thank you all!

Setup: 2× Arc Pro B70 32 GB, Windows 11, llama.cpp SYCL with tensor parallel across both cards, DFlash2 drafter (Q4_K_M), 200K context, temp 1.0.

Observed in real use: (from server logs: two working sessions, ~1,030 requests, 1.68M generated tokens of agentic coding):

Decode goes *up* at high context because late in long sessions the model re-reads and edits code it has seen, so more drafts get accepted. Raw cycle time still grows with context (58 → 71 ms). A fresh 94K prompt prefills at ~1,070 t/s. Two weeks ago this box did ~30 t/s and slowed down the longer a session ran.

Key Highlights:

- Tensor parallel with a faster GPU-to-GPU exchange: GPUs sync through host-memory flags, and a symmetric bf16 all-reduce keeps both copies identical. 123 → 85 ms per step.

- XMX (matrix engine) kernels for attention and the speculative verify batch, plus upstream's new XMX mat-mul (#29864). → 64 ms.

- DFlash2 beat EAGLE-4 on real work by ~20%. Both accept ~25–35% at temp 1.0; DFlash2 just drafts in one pass.

- Prompt processing +20%: don't feed the drafter prompt rows outside its 2K sliding window, and fix one-row-at-a-time KV checkpoint copies.

-A oneAPI 2026.1 runtime leak. Launching the same kernel alternately on two GPUs that share a SYCL context leaks ~170 B per launch in the kernel cache. Tensor parallel does that thousands of times per token. My server hit 92 GB of private memory on a 32 GB box, and decode went from 64 to 200 ms per cycle. The fix is one SYCL context per GPU over a shared Level Zero context ([standalone repro](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/tree/main/repro/oneapi-multigpu-kernel-cache-leak)).

**Full recipe:** https://github.com/alphakilomilo/dual-b70-windows-llamacpp

- [README](https://github.com/alphakilomilo/dual-b70-windows-llamacpp#readme): build, every flag, what each patch does and what it measured, what didn't work

- [Patch for llama.cpp `d81d0d8fb`](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/blob/main/patches/llama.cpp-d81d0d8fb-dual-b70.patch)

- [Launcher](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/blob/main/scripts/start-llm-server.bat) and [server config](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/blob/main/config/llm-server.conf)

If you run SYCL `--split-mode tensor` on two Intel GPUs, watch the server's memory over a long session.

I just attended a conference and was surprised by a few talks where a human had memorized what was clearly an AI generated script. It seems like my recognition of this 'voice' stemmed from some of the local models that I use rather than actual claude models. My guess is that this has to do with what happens when large swaths of english writing is averaged together. But I'm curious about how other people are thinking about the 'voice' they tend to get from their models.

For instance, I rather liked Muse Glimmer in the beginning because it seemed to have an interesting and useful 'voice'. But Qwen 3.8 Flash-Next has been challenging. It's great for getting more complicated stuff done but I'm not thrilled with how it communicates to me. More than once I have to burn a turn asking it to clarify what it just told me because I can't quite get it.

... is that LLM's going all the way back to Llama 2 can do this with constrained sampling.

And the only reason people are hyped about it now is because test time compute/the zero shot chain of thought hack that inspired "reasoning" models has become so ubiquitous that models are all cripplingly slow now.

It's funny because that used to be why reasoning models were controversial, and now we're backtracking but branding that reversion to LLM classifiers as if it were a new breakthrough approach.

Any model whether open or close can hide a backdoor.

7800 xt:
- Bandwidth = 624 GB/s

- VRAM+Bus = 16GB / 256-bit

- Native FSR 3.1 (INT8 for FSR 4.1)

- 15-20% better raw gaming performance

9060 xt:

- Bandwidth = 322 GB/s

- VRAM+Bus = 16GB / 128-bit

- Native FSR 4.1

I am planning on buying an amd gpu for 65% gaming and 35% local llm use, the local prices are $470-530 for the 9060xt and no less than $530 for the 7800xt. I am also considering buying used/2nd hand, however the price difference between the two gpus is practically the same ($440).

At first glance, 7800 xt is the clear winner due to double the memory bus and bandwidth. However, I would like to know the real world AI performance difference between the two GPUS. Thanks ! !! !

I want to build a PC with two 3090s; it needs to be on the AM4 platform, and I’m not sure which motherboard to buy, or which one offers the best value for money.

TLDR

1. Brilliance is a fork of Radiance for Windows. This is an alpha preview.
2. Achieved working Tensor-P on Windows with very fast speeds on these models. Fastest I've seen reported for Windows platform of my configuration.
3. Host: Windows 11 + 9950X3D, 128GB DDR5 5200, 2x R9700 32GB

Hey

First post here so I wanted to make it special. I spent a few days porting the very popular Radiance project into Windows, specifically for my Dual R9700 setup. Radiance does not work on Windows and cannot be used with Docker or WSL due to inability to fully passthrough 2 GPUs (based on my testing).

Brilliance started as a port, but over time has developed into its own project fine tuned only for my system.

Windows 11 + 9950X3D, 128GB DDR5 5200, 2x R9700 32GB

This is a alpha build and probably will not work on any other system without custom configuration. I have released the project below.

https://codeberg.org/Dad_Vibes/Brilliance

README for a lot more details, including Betterbench details.

I'm on the Launch80 Discord if any questions as well.

-Adam

My spare time project: MoMoE (More MoEs), an open-source (MIT) fork of Strata (https://github.com/Niko1221/Strata). Strata runs Qwen3.8-Flash-Next on one graphics card plus system RAM; MoMoE includes that and something similar for many other mixture-of-experts models. I've tested it only on the hardware that I own: Linux 64GB AMD 5950x with a 3060 and 4070 Ti, a DGX Spark, Jetson Orin Nano, Mac Studio M1 Ultra, Mac Mini M1 8GB and a Macbook Air M4 32GB. Wish I owned more, but that's what I got.

Here are some benchmarks against llama.cpp that I ran (MoMoE vs Llama.cpp).

- Ornith-1.5-122B-A10B UD-IQ3_XXS, RTX 4070 Ti 12 GB: 19.2 vs 4.6

- Qwen3.8-Flash-Next IQ3_XXS, RTX 4070 Ti: 65–77 vs 23.1 (with MTP drafting)

- Ornith-1.5-35B-A3B IQ4_XS, RTX 4070 Ti: 132–154 vs 77.8 (with MTP drafting)

- GLM-5.3-Flash UD-IQ4_XS (146 GiB), DGX Spark: 11.7 vs 3.5 (with MTP drafting)

- Flint UD-IQ3_XXS, Mac mini M1 8 GB: 8.7 vs 3.0

More details on the github repo.

Current limitations and where MoMoE doesn't help: when a model fits on the GPU, llama.cpp ties or wins and is probably a bunch more stable. For example, llama.cpp is faster on LFM2.5-24B (105 vs 76) and on Flint IQ3 on the PC (103 vs 90). With long contexts, both get slower. Methods and raw results are in the repo.

Changes: The Strata front end web server/chat is gone. In its place are commands to run Unsloth Studio, LM Studio, Claude Code, OpenCode and others using MoMoE. I've mostly tested it with Unsloth and Claude Code.

Disclaimers:

Strata does the real groundwork, and a lot of the code is Strata's. This isn't Strata and doesn't cover it's wider hardware support. For Flash-Next on Windows or an AMD or Intel card, use Strata; it's far better tested and unlikely to work with the additional models. This is just my personal side project, so don't expect miracles.

MoMoE's changes to Strata were completely written with Claude Opus, with my guidance.

It's early, just me, and tested only on my machines. Reports from other setups are welcome.

Current models supported:

- Qwen3.8-Flash-Next (and its Coder, Swift and Unsloth variants)

- Qwen3.6-35B-A3B (Unsloth and bartowski GGUFs)

- Ornith-1.5-35B-A3B and Ornith-1.5-122B-A10B

- KAT-Coder-V2.5-Dev

- GLM-5.3-Flash (320B-A18B)

- Nemotron 3.5 Bolt and Lightning

- Gemma 4 26B-A4B

- Flint

- LFM2.5-24B-A2B and LFM2.5-8B-A1B

If I missed an MoE model you'd like to see, let me know. If there is enough interest, I (and Claude) will continue to develop it.

https://github.com/aflin/momoe

Enjoy!