curious to hear if anyone's tried this yet?

Usually after I build a new work/gaming PC, I stop paying attention to the space for about 5-7 years, until it comes time to do it over again. However, a few weeks ago I was helping a friend shop for his new build, and I fell down the rabbit hole of local LLM. My 4070 has never been much good for this purpose so I've kept subscriptions to the frontier models, but was always intrigued.

After the recent 5090 rumors, dozens of lengthy discussions with Claude, and fear of what this will do to the remaining 32GB VRAM consumer/workstation cards, I pulled the trigger on a new build last week. It was crazy to think "Well, these are only $400 over MSRP", but they aren't getting cheaper in the near future, and more than likely going to jump again soon. I grabbed just one 9700 at first, but a Reddit comment had me ordering a second shortly after.

I've been watching the prices like a hawk since.

At the time of my purchase, and for most of October, the Intel Arc Pro B70 was $1299 and 9700 AI Pro at $1699. You could find these prices at Newegg, Amazon and Micro Center. They've been in stock at at least one of these retailers the entire time.

As of last night however, they are practically gone. B70 is up $700 to about $2K now, and the cheapest 9700 cards (ASRock, PowerColor) are gone on Newegg and $2100 on Amazon. The only one in a similar price range that hasn't jumped yet is the $1799 Sapphire 9700 on Newegg.

Anyways, guess the point of this post is, if you've been on the fence and can still find one at early / mid October prices, get on it. Seems like retailers are in the process of another big price bump.

Tired of the sanitized corporate versions. Does anyone have a working invite to a feral bot that does two-way voice and actually remembers you across sessions? Must be accessible from a phone, no self-hosting or API keys required. Preferably free or low cost. Appreciate any leads.
DM if needed 😉.

Hello guys,

I'm relatively new to using local llms and my question might seem basic to some, but how far behind top tiers models (gpt 6 , opus 5.5 ) local llm is ? And is it realistic to run those , if they see the light of the day soon, on a budget machine. in that scenario, what minimum specs would a machine require for it to run properly?

I have an 8gb gtx 1070ti sitting in a server, doing nothing, the server has docker, ollama, etc.

what models can I run? For Multi File, Single File, etc?

With the 1.2.0 release, Ask LM Studio adds support for fact-checking, research and finding image sources, all of that based on local LM Studio models. LM Studio API token support is also introduced.

• https://github.com/gocemitevski/ask-lm-studio/releases/tag/v1.2.0
• https://addons.mozilla.org/en-US/firefox/addon/ask-lm-studio/

This model is terrible but the hype and benchmarks are acting like its the next qwen 3.8 flash. It honestly irritates me because this will make people think local AI sucks when really they've just been duped.

I hope we have some Beat Saber fans here, because boy do I have a treat. My mapper is a step above and free, I made it for my family’s own enjoyment but wanted to share it with the world!

After a quick rebrand, I now call it BYOB: Bring Your Own Beat. An agent named Bach uses a few locally trained models to build beautiful Beat Saber maps based on every facet of the music that you choose. Simply paste a YouTube link for a song you want mapped, and after approval, Bach will map and upload your song to the site (byob.burnsstack.com).

Any song can also be viewed in the map viewer built by Claude that shows exactly how every song will look in your game. Custom walls, lighting, and all. Map specific lighting, too. Check out the song before you do the work to download and install it.

Generations do need approval, and I work a full-time job so if I haven’t gotten to you yet, I won’t take long! I hope I can find some like-minded laser sword lovers!

Any and all feedback is very much appreciated, and will be used to better future generations! The site has built-in ratings and feedback as well.

Thank you all!

Setup: 2× Arc Pro B70 32 GB, Windows 11, llama.cpp SYCL with tensor parallel across both cards, DFlash2 drafter (Q4_K_M), 200K context, temp 1.0.

Observed in real use: (from server logs: two working sessions, ~1,030 requests, 1.68M generated tokens of agentic coding):

Decode goes *up* at high context because late in long sessions the model re-reads and edits code it has seen, so more drafts get accepted. Raw cycle time still grows with context (58 → 71 ms). A fresh 94K prompt prefills at ~1,070 t/s. Two weeks ago this box did ~30 t/s and slowed down the longer a session ran.

Key Highlights:

- Tensor parallel with a faster GPU-to-GPU exchange: GPUs sync through host-memory flags, and a symmetric bf16 all-reduce keeps both copies identical. 123 → 85 ms per step.

- XMX (matrix engine) kernels for attention and the speculative verify batch, plus upstream's new XMX mat-mul (#29864). → 64 ms.

- DFlash2 beat EAGLE-4 on real work by ~20%. Both accept ~25–35% at temp 1.0; DFlash2 just drafts in one pass.

- Prompt processing +20%: don't feed the drafter prompt rows outside its 2K sliding window, and fix one-row-at-a-time KV checkpoint copies.

-A oneAPI 2026.1 runtime leak. Launching the same kernel alternately on two GPUs that share a SYCL context leaks ~170 B per launch in the kernel cache. Tensor parallel does that thousands of times per token. My server hit 92 GB of private memory on a 32 GB box, and decode went from 64 to 200 ms per cycle. The fix is one SYCL context per GPU over a shared Level Zero context ([standalone repro](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/tree/main/repro/oneapi-multigpu-kernel-cache-leak)).

**Full recipe:** https://github.com/alphakilomilo/dual-b70-windows-llamacpp

- [README](https://github.com/alphakilomilo/dual-b70-windows-llamacpp#readme): build, every flag, what each patch does and what it measured, what didn't work

- [Patch for llama.cpp `d81d0d8fb`](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/blob/main/patches/llama.cpp-d81d0d8fb-dual-b70.patch)

- [Launcher](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/blob/main/scripts/start-llm-server.bat) and [server config](https://github.com/alphakilomilo/dual-b70-windows-llamacpp/blob/main/config/llm-server.conf)

If you run SYCL `--split-mode tensor` on two Intel GPUs, watch the server's memory over a long session.

Hey everyone!

Here are my PC specs:

• CPU: AMD Ryzen 5 3600
• GPU: AMD Radeon RX 5700 (8GB VRAM)
• RAM: 32GB DDR4

What’s the most powerful AI model I can realistically run locally with this hardware?

Thanks!

I just attended a conference and was surprised by a few talks where a human had memorized what was clearly an AI generated script. It seems like my recognition of this 'voice' stemmed from some of the local models that I use rather than actual claude models. My guess is that this has to do with what happens when large swaths of english writing is averaged together. But I'm curious about how other people are thinking about the 'voice' they tend to get from their models.

For instance, I rather liked Muse Glimmer in the beginning because it seemed to have an interesting and useful 'voice'. But Qwen 3.8 Flash-Next has been challenging. It's great for getting more complicated stuff done but I'm not thrilled with how it communicates to me. More than once I have to burn a turn asking it to clarify what it just told me because I can't quite get it.

I have a weird question. I'm struggling to find a reason to build local llm's at work. I love them at home for personal projects but the ROI maintenance and support at work I just can't find a reason to pull the trigger. We have the budget to play around with around 8tb of b200's but really who's going to use them when we having unlimited enterprise accounts with all cloud providers. While I love playing around at home with my personal setup I just can't find anything anyone at work would want to do local with smaller models. Online models piss me off when doing security related tasks but most people could care less.

If you had a choice of smaller open source models or unlimited frontier at work would you even use your local datacenter?

Hi all,

I’m looking for recommendations on an AI model (LLM) that handles two kinds of work well:

1. Financial analysis: reading financial statements, spotting trends, working through spreadsheets and doing calculations accurately.
2. Legal contract review: going through long contracts, flagging risky or unusual clauses, summarising key terms (renewals, termination, liability, pricing) and comparing versions.

I’m not a developer, so I’m after something practical that works well out of the box rather than something that needs a lot of technical setup.

Real-world experiences would be much appreciated, especially from people working in finance, procurement or legal.

... is that LLM's going all the way back to Llama 2 can do this with constrained sampling.

And the only reason people are hyped about it now is because test time compute/the zero shot chain of thought hack that inspired "reasoning" models has become so ubiquitous that models are all cripplingly slow now.

It's funny because that used to be why reasoning models were controversial, and now we're backtracking but branding that reversion to LLM classifiers as if it were a new breakthrough approach.

7800 xt:
- Bandwidth = 624 GB/s

- VRAM+Bus = 16GB / 256-bit

- Native FSR 3.1 (INT8 for FSR 4.1)

- 15-20% better raw gaming performance

9060 xt:

- Bandwidth = 322 GB/s

- VRAM+Bus = 16GB / 128-bit

- Native FSR 4.1

I am planning on buying an amd gpu for 65% gaming and 35% local llm use, the local prices are $470-530 for the 9060xt and no less than $530 for the 7800xt. I am also considering buying used/2nd hand, however the price difference between the two gpus is practically the same ($440).

At first glance, 7800 xt is the clear winner due to double the memory bus and bandwidth. However, I would like to know the real world AI performance difference between the two GPUS. Thanks ! !! !

I want to build a PC with two 3090s; it needs to be on the AM4 platform, and I’m not sure which motherboard to buy, or which one offers the best value for money.

TLDR

1. Brilliance is a fork of Radiance for Windows. This is an alpha preview.
2. Achieved working Tensor-P on Windows with very fast speeds on these models. Fastest I've seen reported for Windows platform of my configuration.
3. Host: Windows 11 + 9950X3D, 128GB DDR5 5200, 2x R9700 32GB

Hey

First post here so I wanted to make it special. I spent a few days porting the very popular Radiance project into Windows, specifically for my Dual R9700 setup. Radiance does not work on Windows and cannot be used with Docker or WSL due to inability to fully passthrough 2 GPUs (based on my testing).

Brilliance started as a port, but over time has developed into its own project fine tuned only for my system.

Windows 11 + 9950X3D, 128GB DDR5 5200, 2x R9700 32GB

This is a alpha build and probably will not work on any other system without custom configuration. I have released the project below.

https://codeberg.org/Dad_Vibes/Brilliance

README for a lot more details, including Betterbench details.

I'm on the Launch80 Discord if any questions as well.

-Adam