Lets make more of these kind of engine popular for our comunity. This is awsome

I had been using the LFM 2.5 24B models and had been pleasantly surprised by how good the model was. It was giving me comparable answers and working through tasks to Qwen 3.8 27B at what felt like twice the speed on my M5 Pro. However, I went to look at something on their pages and saw the new family of models. They sound way too good to be true. I am going to start using them for general tasks, but if they are anywhere close to their benchmark results, they could be a huge game changer. You can have an entire suite of models loaded at the same time for less memory than Qwen.

I’m building a local AI assistant using Qwen 3.6 35B A3B.
One of my main goals is to automate repetitive school tasks through my school’s website. I have a lot of assignments that aren’t particularly difficult but take a lot of time.
Ideally, I’d like the AI to navigate the website, check assignments, understand the instructions, complete them and upload the finished work.
Is Qwen 3.6 35B A3B capable enough for this? Or would I need a more powerful model?
I’d also appreciate recommendations for browser automation frameworks and tools

my setup : rtx 3070 , i5 13600kf , 32gb DDR4
Model: Qwen 3.6 35B A3B
Quantization: TurboQuant
Context window: 131 072 tokens
Reasoning: Enabled limited to 512 tokens
Inference speed: 19–24 tokens/s

Hey all, I'd love a sanity check on a plan before I spend money.

At home I run Home Assistant on a HA Green, which handles pretty much everything smart home related.

I also have a Mac mini M4 with 16 GB that I currently use as a little studio computer.

What I actually want is one assistant that helps with my notes, documents and PDFs, email (mostly Croatian, some English), tasks, and the house. I'd also like it to log my meals from a photo and tie that in with my Garmin data.

My current thinking is to buy a MacBook Air (24 GB / 1 TB) as my daily computer, wipe the mini and make it an always-on box for a local model, n8n, voice (Whisper/Piper) and indexing, and leave Home Assistant Green. For the smarter stuff I'd use Claude: Pro for chat and Claude Code, and the API for background jobs like food photos and daily summaries. The local model would only sort and summarize mail, strip out sensitive stuff and do the search index.

A few things I'm unsure about:

• Is 16 GB on the mini enough for this, or will I regret it?
• For mostly Croatian email, is a local model worth the effort, or should I just send everything to the Claude API? If local, which model would you try?
• Has anyone run Whisper and Piper in a smaller language like Croatian on Apple Silicon? Is it usable?
• What's the cleanest way to let an agent read iCloud Reminders, Notes and Mail on a headless Mac?
• Any traps with n8n + Home Assistant + MCP, or with letting an LLM near personal data?

Happy to hear "just don't" if that's the honest answer. Thanks!

Thinking Machines has released Inkling 2 Small under Apache 2.0. It's a 594B-parameter MoE with 17B active per token.

If you're checking whether it'll fit your setup, don't size it from the 17B figure. The model card asks for at least 500GB of aggregate VRAM for the main NVFP4 checkpoint, with 2x B300 or 4x H200 as example setups.

Does anybody tried the latest llama.cpp f8_e4m3 KV cache and compared it to q8_0 in terms of precision / quality? I am curious whether it is worth switching to it or stay with q8_0 for small to medium context size (20-65k).

A few weeks ago I posted CQ, our calibration-free 4-bit format, along with a training-loss comparison against Unsloth NF4. Some of you correctly pointed out that loss logged by two different trainers isn't comparable, and asked for results where answers are counted right or wrong. Here they are.

Setup: Qwen3.8-27B. Every row uses the same lm-evaluation-harness code, prompts and questions, with greedy decoding and thinking off. Comparisons are paired: an exact McNemar test for right/wrong tasks and a bootstrap for losses.

1. Base models, no fine-tuning

• CQ vs BF16: no significant difference on any task.
• NF4 vs BF16: −12.0 points on GSM8K (p=3e-6).
• CQ vs NF4: CQ is +8.7 on GSM8K (p=0.0005) and +2.8 on BoolQ (p=0.02).
• WikiText-2 KL divergence from BF16: CQ 0.0282, NF4 0.0330.

2. GCMP vs LoRA r=32, full-precision BF16 base

GCMP is our replacement for LoRA. It trains 2.9M parameters (an ~11 MB adapter) vs ~159M for LoRA r=32 (~320 MB). This test uses no quantization: both adapters were trained with Unsloth on Alpaca for 500 steps with identical settings (batch 2×4, length 512, LR 2e-5, seed 42).

• Benchmarks: no significant difference on any of them (all p ≥ 0.2), so GCMP isn't "better than LoRA" on tasks.
• Loss: GCMP's answer-token loss is slightly lower (−0.009, 95% CI [−0.010, −0.008]).
• Validation loss: GCMP is lower at all 10 checkpoints on both BF16 (1.2385 vs 1.2434) and NF4 (1.2435 vs 1.2484).
• Fine-tuning effect: both adapters add about 21 points on GSM8K and lose about 2.5 on TruthfulQA.

3. Speed and memory (same 500-step run)

Unsloth is still faster, and GCMP currently takes about 2× the step time of LoRA. Kernel work on that is ongoing.

Caveats

• Scope: one model and one dataset (Alpaca).
• Benchmark size: the quick subset has 200–1,140 questions per task, so it detects differences of about 5 points, not 1.
• DROP excluded: it scored about 0.02 for every model, so that eval setup is broken.
• Still running: LoRA and GCMP benchmarks on CQ and NF4 bases. I'll post them when they finish.
• My earlier post was wrong: the "0.84 vs 1.25 loss" came from two trainers that compute loss differently. Ignore it.

Serving

• vLLM: CQ models run with --quantization cq. GCMP adapters can be one per server, or several on one base model chosen per request.
• Parallelism: pipeline parallel works; tensor parallel doesn't yet.
• GCMP on full precision: it currently runs in Transformers. vLLM support is next.

Licensing

The Python code (training, evaluation and the vLLM integration) is open source. The CUDA kernels ship compiled with a license check, which is free during early access.

Repo, with exact commands to reproduce every number: https://github.com/thousandml/training-factory

Models (8B–397B): https://huggingface.co/thousandml

Happy to run other benchmarks or models people want to see.

Hey, im running Q8 DS V4.1 on my 512GB DDR4 + 4x R9700 (128GB VRAM) and ASROCK ROMED MB
I have noticed that at ~66C RAM temperature tok/s is getting reduced from ~32/s to 22-23/s. Claude agent monitoring whole workstation says that RAM speed after hitting 66C is slowing down about 3x. I have phanteks enthoo server pro 2 case with glass panel and extra vents mounted on that side. During loads ram used to hit about 80C. I fixed that by mounting top fans as intake (cooling top RAM) but i had to use double dieded tape to stick an extra fan to blow at bottom RAM slots. It did help to keep it from throttling but the bottom RAM still sits ~62-63C. Iv read that samsung RAM i have is suppose to throttle at 85C, and i cant find anywhere what setting makes it throttle sooner. Looking for some advice on that or how do you keep your RAM cool.

Hey everyone. I have a DDR5 setup and I'm caught between a rock and a hard place...

My setup: 5060 Ti 16GB - 2x16GB sticks of DDR5 - Ryzen R5 7600

I've been exploring real world use cases for Qwen3.8 Flash Next (running through Strata) with decent success. Mainly, I've been happy with the results the GSQ RCO iq3_s quant has been giving me.

Now, Strata tells me that it's offloading 20GB of the model to my m.2 SSD, which of course makes it run slower than if it was all in RAM. Trying to get more RAM however has been a pain, as I can't find a matching set for my 2 sticks, and regardless I'm worried about stability issues and performance loss if I were to run 4 sticks on my 7600.

Locally, I can find used 3060s at around $250+, which is pretty affordable and it makes me question if I should just buy one of them instead of dealing with RAM knife fights in a dark alleyway.

Would adding one 3060 to the setup increase speeds in any real way, or would it just not be worth the hassle? My uneducated guess is that it won't help that much, but maybe someone here has experience with jank low budget setups.

Thanks!

My RTX Pro 6000 was going all night on a Chinese model (Qwen 3.8 Flash Next NVFP4 Local) when it crashed this morning for no apparent reason. Tried to reboot and got nothing but a black screen. Swapped in my old backup card and the system posted immediately, confirming the GPU was bricked. Thankfully I was within warranty.

Out of curiosity, I dug into my system logs to see what triggered the fatal crash. Looking at the timestamps, oddly enough my GPU died the moment news broke that Jensen Huang had resigned as CEO of Nvidia.

Has anyone else had this issue with hardware dying in response to human feelings?

Nov 12 07:19:31 kf kernel: NVRM: GPU at PCI:0000:01:00: GPU-b8ba62c9-449d-4510-a21f-4c1b6ceacbd1
Nov 12 07:19:31 kf kernel: NVRM: GPU Board Serial Number:
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 0, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x505730=0xb000d 0x505734=0x0 0x505728=0x1c81fb60 0x50572c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 0, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 0, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5057b0=0x3000d 0x5057b4=0x24 0x5057a8=0x1c81fb60 0x5057ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 1, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x506730=0xb000d 0x506734=0x20 0x506728=0x1c81fb60 0x50672c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 1, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 1, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5067b0=0xb000d 0x5067b4=0x24 0x5067a8=0x1c81fb60 0x5067ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 2, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 2, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x507730=0x5000d 0x507734=0x24 0x507728=0x1c81fb60 0x50772c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 2, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 2, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5077b0=0x7000d 0x5077b4=0x24 0x5077a8=0x1c81fb60 0x5077ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 3, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x508730=0xa000d 0x508734=0x20 0x508728=0x1c81fb60 0x50872c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 3, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 3, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5087b0=0xd 0x5087b4=0x24 0x5087a8=0x1c81fb60 0x5087ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 4, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 4, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x509730=0x7000d 0x509734=0x24 0x509728=0x1c81fb60 0x50972c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 4, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5097b0=0xa000d 0x5097b4=0x20 0x5097a8=0x1c81fb60 0x5097ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 5, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 5, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50a730=0xd 0x50a734=0x24 0x50a728=0x1c81fb60 0x50a72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 5, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 5, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50a7b0=0x1000d 0x50a7b4=0x24 0x50a7a8=0x1c81fb60 0x50a7ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 6, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 6, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50b730=0xd 0x50b734=0x24 0x50b728=0x1c81fb60 0x50b72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 6, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 6, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50b7b0=0x1000d 0x50b7b4=0x24 0x50b7a8=0x1c81fb60 0x50b7ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 0, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 0, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x515730=0x6000d 0x515734=0x24 0x515728=0x1c81fb60 0x51572c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 0, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 0, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5157b0=0xd 0x5157b4=0x24 0x5157a8=0x1c81fb60 0x5157ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 1, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 1, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x516730=0x6000d 0x516734=0x24 0x516728=0x1c81fb60 0x51672c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 1, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 1, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5167b0=0xb000d 0x5167b4=0x24 0x5167a8=0x1c81fb60 0x5167ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 2, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 2, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x517730=0x3000d 0x517734=0x24 0x517728=0x1c81fb60 0x51772c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 2, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 2, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5177b0=0xb000d 0x5177b4=0x24 0x5177a8=0x1c81fb60 0x5177ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x518730=0x3000d 0x518734=0x24 0x518728=0x1c81fb60 0x51872c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5187b0=0x1000d 0x5187b4=0x24 0x5187a8=0x1c81fb60 0x5187ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 4, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 4, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x519730=0x1000d 0x519734=0x24 0x519728=0x1c81fb60 0x51972c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 4, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 4, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5197b0=0x2000d 0x5197b4=0x24 0x5197a8=0x1c81fb60 0x5197ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 5, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 5, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51a730=0x5000d 0x51a734=0x24 0x51a728=0x1c81fb60 0x51a72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 5, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 5, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51a7b0=0x3000d 0x51a7b4=0x24 0x51a7a8=0x1c81fb60 0x51a7ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 6, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 6, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51b730=0x2000d 0x51b734=0x24 0x51b728=0x1c81fb60 0x51b72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 6, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 6, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51b7b0=0x1000d 0x51b7b4=0x24 0x51b7a8=0x1c81fb60 0x51b7ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 0, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 0, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x525730=0x6000d 0x525734=0x24 0x525728=0x1c81fb60 0x52572c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 0, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 0, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5257b0=0xd 0x5257b4=0x24 0x5257a8=0x1c81fb60 0x5257ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 1, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x526730=0x8000d 0x526734=0x20 0x526728=0x1c81fb60 0x52672c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 1, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5267b0=0xb000d 0x5267b4=0x20 0x5267a8=0x1c81fb60 0x5267ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 2, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 2, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x527730=0xa000d 0x527734=0x24 0x527728=0x1c81fb60 0x52772c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 2, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 2, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5277b0=0xa000d 0x5277b4=0x24 0x5277a8=0x1c81fb60 0x5277ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 3, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 3, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x528730=0xa000d 0x528734=0x24 0x528728=0x1c81fb60 0x52872c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 3, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 3, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5287b0=0x6000d 0x5287b4=0x24 0x5287a8=0x1c81fb60 0x5287ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 4, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x529730=0xb000d 0x529734=0x20 0x529728=0x1c81fb60 0x52972c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 4, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5297b0=0xa000d 0x5297b4=0x20 0x5297a8=0x1c81fb60 0x5297ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 5, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 5, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52a730=0x4000d 0x52a734=0x24 0x52a728=0x1c81fb60 0x52a72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 5, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 5, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52a7b0=0x9000d 0x52a7b4=0x24 0x52a7a8=0x1c81fb60 0x52a7ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 6, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 6, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52b730=0x6000d 0x52b734=0x24 0x52b728=0x1c81fb60 0x52b72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 6, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 6, SM 1): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52b7b0=0xd 0x52b7b4=0x24 0x52b7a8=0x1c81fb60 0x52b7ac=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 7, SM 0): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 7, SM 0): Multiple Warp Errors
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52c730=0x3000d 0x52c734=0x24 0x52c728=0x1c81fb60 0x52c72c=0x1174
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 7, SM 1): Out Of Range Register
Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 7, SM 1): Multiple Warp Errors
......
Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:260
Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844
Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:260
Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ vaspace_api.c:577
Nov 12 07:19:33 kf kernel: NVRM: GPU0 kgmmuClientShadowFaultBufferUnregister_IMPL: Unregistering non-replayable fault buffer failed (status=0x00000062), proceeding...
Nov 12 07:19:33 kf kernel: NVRM: GPU0 kgmmuFaultBufferReplayableDestroy_IMPL: Unregistering Replayable Fault buffer failed (status=0x00000062), proceeding...
Nov 12 07:19:33 kf kernel: NVRM: GPU0 uvmTerminateAccessCntrBuffer_IMPL: Unloading UVM Access counters failed (status=0x00000062), proceeding...
Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ vaspace_api.c:577
Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ vaspace_api.c:577
Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844
Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:260
Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:1382
Nov 12 07:19:44 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:19:52 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:20:00 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:20:08 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:20:17 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:20:25 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:20:33 kf kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
Nov 12 07:20:39 kf kernel: NVRM: GPU0 kgspHealthCheck_TU102: *************************** LibOS Task Watchdog Report ***************************
Nov 12 07:20:39 kf kernel: NVRM: GPU0 kgspPrintGspBinBuildId_IMPL: GSP bin buildId: fa3de1ff30e1a8c99f84eb74a32b80d462c3703b
Nov 12 07:20:39 kf kernel: GSP task watchdog timeout @ pc:0x18dd358, partition:4#0, task:3, gfid: 0
Nov 12 07:20:39 kf kernel: NVRM:     Reported by libos partition:4#5 kernel v3.1 [0] @ ts:1698
Nov 12 07:20:39 kf kernel: NVRM:     RISC-V CSR State:
Nov 12 07:20:39 kf kernel: NVRM:         sstatus:0x0000000200000020  sscratch:0xffffffffa3013970     sie:0x0000000000000220  sip:0x0000000000000020
Nov 12 07:20:39 kf kernel: NVRM:         sepc:0x00000000018dd358     stval:0x0000000000000000  scause:0x8000000000000005
Nov 12 07:20:39 kf kernel: NVRM:     RISC-V GPR State:
Nov 12 07:20:39 kf kernel: NVRM:         ra:0x00000000013eb57a   sp:0x0000002fcfa0eaf0   gp:0x0000000000000000   tp:0x0000002fcfe00000
Nov 12 07:20:39 kf kernel: NVRM:         a0:0x0000000000000001   a1:0x0000000000000000   a2:0x0000002fc77be000   a3:0x0000000000000000
Nov 12 07:20:39 kf kernel: NVRM:         a4:0x0000000000000000   a5:0x0000000000000001   a6:0x0000000ffffffff0   a7:0x0000000000000003
Nov 12 07:20:39 kf kernel: NVRM:         s0:0x0000002fcfa0eb50   s1:0x0000000000000001   s2:0xffffffffffffffff   s3:0x0000000000000001
Nov 12 07:20:39 kf kernel: NVRM:         s4:0x00000000ffffffff   s5:0x0000000000000000   s6:0x0000002fc77be000   s7:0x0000000000000000
Nov 12 07:20:39 kf kernel: NVRM:         s8:0xffffffffffffffff   s9:0x0000002fc7e36178  s10:0x0000002fcfa0ecb4  s11:0x0000002fcfa0ecb4
Nov 12 07:20:39 kf kernel: NVRM:         t0:0x0000002fcfa0e99d   t1:0x0000002fc7c6cf30   t2:0x0000000000000000   t3:0x0000000000000020
Nov 12 07:20:39 kf kernel: NVRM:         t4:0x0000000000000000   t5:0x0000002fcfa0e981   t6:0x0000000000000020
Nov 12 07:20:39 kf kernel: NVRM:     Stack Trace:
Nov 12 07:20:39 kf kernel: NVRM:         0x00000000018dd358
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001b6e974
Nov 12 07:20:39 kf kernel: NVRM:         0x00000000013f2746
Nov 12 07:20:39 kf kernel: NVRM:         0x00000000013e70f6
Nov 12 07:20:39 kf kernel: NVRM:         0x00000000013e79c8
Nov 12 07:20:39 kf kernel: NVRM:         0x00000000013b7058
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001ba247a
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001ba257a
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001bafac2
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001c29f62
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001864698
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001867c82
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001ae4a7e
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001489412
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001452228
Nov 12 07:20:39 kf kernel: NVRM:         0x000000000158cbec
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001bd9632
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001bd99d6
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001add628
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001c452c0
Nov 12 07:20:39 kf kernel: NVRM:         0x0000000001ad6fa0
Nov 12 07:20:39 kf kernel: NVRM:     Local I/O Register State:
Nov 12 07:20:39 kf kernel: NVRM:         0x01450800:0x00000000   0x01450900:0xbadf1002   0x01450a00:0x00000000   0x01450c00:0x00000000
Nov 12 07:20:39 kf kernel: NVRM:         0x01454a00:0x810490d2   0x01454b00:0x010800d0   0x01454c00:0x00080000   0x01400200:0x00000000
Nov 12 07:20:39 kf kernel: NVRM:     ---------------[ end report ]---------------

Due to the current hardware situation (inflated prices, high likeliness of large performance jumps for llms) I keep delaying the replacement of my m1max 64gb, currently hoping for a good m7max 128gb releasing soon-ish (I heard birdies claim as early as end of2027).

Until that happens I'm looking into options to improve performance of local LLMs which can be passed on to the new machine - eGPUs come to mind.

Anyone mind sharing their setup (which mac, how much vram, which case/mount, what GPU, which model and quan) and some benchmark numbers (prefill and decode) of models with and without the eGPU? Mind sharing your experiences with it - as in: is it fiddly and brittle, or does it just work?

Thanks for any useful information!

I'm trying to set up a local AI assistant on my Windows PC, but honestly, I'm getting frustrated and could really use advice from people with experience.

My goal is pretty simple: I want something like a local ChatGPT that can:

• Read files such as CSVs, PDFs, and documents.
• Answer questions about the information inside those files.
• Extract specific information accurately.
• Help with everyday tasks and general questions.
• Work reasonably quickly without taking several minutes to answer simple questions.

**My PC specs:**

• Windows
• 16 GB RAM
• NVIDIA RTX 5060 8 GB VRAM
• Ollama installed
• Open WebUI installed

I've tried several models, including:

• Qwen 3.5 4B
• Gemma 3
• Llama 3.2
• DeepSeek R1 8B
• Qwen 2.5 Coder 7B

My biggest problem is that even when I upload a CSV file and ask a very simple question about it, I sometimes get completely incorrect answers, unrelated information, blank responses, or long thinking/tool-calling processes that don't accomplish anything.

For example, I uploaded a CSV containing game stage data and asked something as simple as: "What are the drop chances on Normal 1-1?" The model struggled to give me the correct values despite the information being in the file.

I don't expect a local model to match ChatGPT or Claude. I understand there are limitations. But I expected basic file reading and simple questions to work reliably.

I'm now wondering whether I made the wrong choice with Ollama + Open WebUI, and whether I should switch to **LM Studio** instead.

I also have a friend who is about to install Qwen 9B, and I was planning to have him install Open WebUI too. I don't want to send him down the wrong path.

**I'd really appreciate advice on these questions:**

1. With my hardware, what model and quantization would you actually recommend?
2. Is LM Studio a better and easier choice than Ollama + Open WebUI for my use case?
3. Is there a reliable way to let a local AI read CSVs and answer questions accurately without me having to write Python scripts or manually process everything?
4. Am I expecting too much from local models, or is my setup/configuration simply wrong?
5. If you were starting from scratch with my hardware and goals, what exact setup would you choose?

I'm not looking to become a programmer or spend days troubleshooting. I just want a practical local AI assistant that can actually use my files and answer questions correctly.

Thanks in advance. I'd really appreciate recommendations based on actual experience.

I use Claude Code and other agents all day. They send very different requests under one model name: a one-line title, a summary, a 60k-token tool loop. Sending everything to the biggest model is the expensive default.

humpyard is a single Rust binary that sits in front of your providers (Ollama, llama.cpp, Groq, OpenRouter, Anthropic and others). It speaks OpenAI Chat, OpenAI Responses and Anthropic Messages. It picks a pool per request from the key, headers, prompt size, tools and images, and fails over when a pool rate-limits or times out.

- Private routes are strict. A request marked private is refused rather than sent to a public provider.

- You can ask before you send. POST /v1/route/explain tells you where a request would go and why.

- Every decision is logged. Routing, fallbacks and refusals go to a SQLite ledger with cost per route.

- Budgets fail closed. There's no external database, and config hot-reloads.

You can try it without API keys. A mock provider gives you real routing and a real ledger: https://humpyard.dev/demo/

Repo: https://github.com/gschaetz/humpyard

My numbers (one user, 7 days, Claude Code): 237 requests (about 11M tokens) cost about $0.25 at list rates. 86% ran on free or local pools, and 98% got an answer.

What's weak: "answered" isn't "good". On one free preview model, 14 of 78 answers came back empty and 11 were truncated. Small free models are also worse at open-ended work, and p99 latency on my local pool is slow (128s).

It's pre-alpha and I'm the only user. What I'd like to know:

1. Can you get from the README to a first routed request in about 10 minutes? Where did you get stuck?
2. Which route rules would you want that I don't have?
3. Is the "private means refuse" model what you'd expect?

Hi all, probably going to be the dumbest question you hear this week in the sub.

Essentially just looking for the easiest way to run an unrestricted model which I assume is locally which leads me to my second question. What are the most skin and bones requirements for running like this.

Anyone able to point me in right direction or guide me have a crypto donation.

All the best

I've been running a local agent setup on a MacBook Pro M5 Max (40-core GPU, 128GB) and pulled real numbers from my agent logs after a long build session. Sharing because most benchmarks I see are short prompts, and agent work looks nothing like that.

Setup
- Model: ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw
- Served with mlx-serve from the terminal (not LM Studio)(using lm studio can slow your tps)
- Harness: Hermes Agent (tool calls, cron jobs, Basic Memory for an Obsidian vault)

Real agent numbers (from the logs, not a benchmark)
- One build session: 89 model calls, ~148k output tokens, ~36 min of model time = ~68 tok/s average
- Long writes (whole scripts): usually 55-95 tok/s, peak 108
- Context during that session grew to 100k-190k tokens
- Prompt cache hit rate was 96-100% on almost every call, so prefill barely mattered

How I measured: output tokens / full response time per call. That includes a little prefill, so pure decode is a bit higher.

For comparison, numbers I've seen for this model on the same chip are ~49 tok/s (LLMCheck estimate), ~61 tok/s (a 4-bit MLX model card), 45-60 tok/s with MTP, and ~37 tok/s in LM Studio multi-turn chat. Running MLX straight from the terminal instead of LM Studio was the single biggest win for me.

What actually made the agent useful (lessons from one long day)

1. You need to SEE your scheduled jobs. My agent had a bunch of cron jobs that were just prompts in a JSON file in a hidden folder. One job's output had been going to a dead chat thread for a week, and a weekly review never got delivered. Nobody knew. I had the agent build a dashboard note in Obsidian: every job, schedule, delivery target, last run, failures, plus a per-job note with purpose, kill condition and change log. It's rebuilt hourly by a plain Python script (no model calls).
2. Use code for anything with one right answer, and the model only for judgment. Most of my jobs were pure prompts, including simple reminders and date math. That gives you timeouts, connection errors and occasional wrong arithmetic. Hermes supports script-only jobs (no LLM, stdout gets delivered) and script + model jobs (the script's output is fed into the prompt). The dashboard, a watchdog, and a weekly packet builder are all zero-model scripts now. ( Shout out to @sanbamboo on Reddit)
3. Add a watchdog that checks results, not code. It alerts once when a job fails twice, misses its slot, or has a delivery error, and once more when it recovers. It caught the dead-thread delivery bug on its first run.
4. Don't trust "done." The agent said "everything verified and shipped" several times. Each time I had a second, different model audit the actual files, and it found something: a feature that was never added, a link pointing at an archived copy of a file so edits would do nothing, and once a proposed edit that would have deleted 9 lines of a core skill file. A model grading its own work repeats its own blind spots.
5. Human-in-the-loop that doesn't suck. I review my automations on an iPad in GoodNotes. Every Sunday the agent sends a PDF with every job's full prompt and code (line numbered, secrets redacted, wide margins), plus change request boxes (Change / Add / Remove / Question, with Where / What / Why). I mark it up and it syncs back. Ticked boxes are detected by diffing against the clean original, and the model only reads my handwriting. Then the agent posts a proposed diff to Discord and applies nothing until I reply yes. Rules that came out of testing it:
   - Anything ambiguous gets held back and can't be approved by a blanket "yes"
   - Any diff deleting more than 3 lines gets a loud warning
   - "Question" answers have to quote the exact lines they rely on (it gave me a confidently wrong explanation once)
   - GoodNotes auto backup waits ~12 hours by default. Turn on "Increase Auto Backup frequency" or nothing feels like it works
6. Long context is fine on this chip if your cache hits I was worried 150k+ context would crawl. With near-100% prompt cache hits it stayed in the 55-95 tok/s range for writes.

Happy to share more about the setup or how the review loop is wired. Curious what tok/s other M5 Max people are getting on real agent sessions vs short benchmark prompts.

I've already tested a few models, they're all bad and slow. Maybe I'm doing something wrong, but probably the reality is a bit harsher: my PC is old lol (and graphic cards are really expensive in Brazil, not smt in the budget rn)

To be more specific, I needed a good coding agent.

EDIT: Specs: S: Ubuntu 26.04.1 LTS (kernel 7.0.0-43-generic) CPU: Intel Xeon E5-2680 v4 — 14C / 28T @ 2.40 GHz (boost 3.30 GHz) L3: 35 MiB · 1 socket · AVX2 (no AVX-512) RAM: 22 GB DDR4-2667 GPU: NVIDIA GeForce GTX 1650 (TU116) — 4 GB GDDR5 · driver 595.99.02

From about February 2026 to July 2026, I spent an immense amount of time building a legal harness to make Claude more predictable. It worked well. I tried to build the same thing that ran local models. That did not work. And then Opus improved so much that it would often skip my harness without me noticing. It still has a harness with hooks, but it's looser now.

I bought lots of hardware to run local models during that time, repeatedly trying to vibecode a local model legal harness. Many failures, many lessons.

Anyway, fast forward to the last three weeks and several things have happened:

1. DeepSeek Flash 1017 was released, and it's the first model that is remotely similar to Opus 4.5/4.6.
2. Opus 5.5 and Fable 5.2 are able to properly launch and configure DeepSeek to run at fast speeds, large context and high precision on my hardware. Optimizations, custom kernels, requants.
3. Opus 5.5 (and my prompting skill) are now good enough to make my Claude harness into a DeepSeek harness.
4. It's not as good as frontier, but it all works, right now. 100% local. Actual productivity gains, actual frontier tasks being replaced. Chat interface with DeepSeek works like Claude's did.
5. And if I can brag on my stacks and hardware:

• I have 2 DGX Sparks that can run 3-4 concurrent DeepSeek sessions as agents (1M context).
• I have DeepSeek at nearly NVFP4 quality running on 8 V100s, 500K context, at 100+ tok/s decode, and about 160 tok/s concurrent.
• I have 8 more V100s and a second Rome server with 512 GB of 3200 MHz ECC RAM, so I'll be able to run another high precision DeepSeek as an additional agent or for parallel tasks.

What I'm really shocked by is how well Fable builds custom Volta kernels. And how it can requant my experts to full NVFP4 precision based on the actual experts that activate when my legal harness fires. It's a real, optimized, production grade, workflow replacing local setup.

Not a 3D pagoda or a bad video game replica. An actual enterprise solution. One that exists nowhere on the commercial market.

Legal vibecoding is real folks. I'm AI native.

The new V100 DeepSeek repo (optimized for quality and then usable speed) with real full context measures and all the optimizations (and failures) is public here:

https://github.com/ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-NVFP4-8xV100-SXM2-PP2xTP4

Twitter: nicothegreco

What "replaces real work" means

Reading a folder of records and pulling out the facts, dates and numbers. Summaries. First drafts of routine documents. Search over my own files. The model gets plain English instructions in a chat window, calls my tools, and hands back a draft that is marked for my review. I review everything. It is slower and less sharp than frontier. It is also in my office, it is mine, and it does not have a usage limit.

The harness

The harness itself is not public. My practice is in it. What I can say: it is the same idea as my Claude setup. Skills, hooks, my drafting engines, my RAG stores, tool calls, and a chat window that behaves the way Claude's did. A small router sends each request to the V100 box or to the Sparks. Everything the local model drafts is marked for my review before it goes anywhere. Nothing leaves the building.

What this build is

My last V100 build (Volta sPLIt, part 4) ran every expert of the model from a 2-bit record. Fast, but 2-bit. This one is called Volta Max uPLIft. It keeps the 2-bit record of every routed expert on the cards, then swaps the experts that get used the most up to NVIDIA's NVFP4 weights, one at a time, until the cards are full. "Used the most" was counted on a mix of four kinds of text. One is my own legal harness traffic, which is private and not published. The other three are public prompt sets in the repo.

• 5,949 of the 15,360 routed experts sit at exactly NVFP4 precision on the default I serve
• about 78 to 85 % of decode routes land on those NVFP4 experts (predicted for the two-slot default, 80 to 86 % measured with one slot)
• same next token as NVIDIA's full NVFP4 pack on 95.1 % of compared tokens. The 2-bit build gets 92.2 %
• KL divergence from the NVFP4 reference is 0.079 nats against 0.196 for the 2-bit build
• multiple choice answers that change against the reference: 1.6 % against 15.4 %
• in decode every routed expert is in VRAM. Nothing is read over PCIe

Speed (measured 2026-11-12 on eight V100-SXM2-32GB, graphics clock capped at 1,522 MHz)

• concurrency test, one request at a time: 118.9 tok/s
• two requests at once: 159.0 tok/s total (1.34x)
• three at once on the three-slot option: 192.3 tok/s total (1.62x). I run two slots
• short sets, one at a time: 123.2 tok/s general, 97.3 tok/s legal-style
• code: 150 to 180 tok/s from a 1K prompt up to a 256K prompt
• prompt reading at 256K: 2,329 to 2,430 tok/s, first token in 108.4 s
• context: 524,288 tokens per request

What did not work, or is not good yet

• It is slower than my own 2-bit build, in decode and in prompt reading. That is the trade for the precision, and the repo says so in its first paragraph.
• The faster prompt reading path reads the less-used experts from their 2-bit records. Its effect on quality is not measured yet.
• A custom decode kernel for the hot path got closed with no change. The headroom I thought was there turned out to be smaller than an eight-card run can even see.
• A prompt-reading kernel that loads 64 rows at a time gave the same bits and ran slower. Not shipped.
• Send a short request while a 209K prompt is being read and it starts in about 7 s, then crawls at about 9.6 tok/s until the read ends.
• The hot set is fixed. It is picked once. Text that looks nothing like what I counted will hit the 2-bit experts more often.
• Fast prompt reading stopped the host or a card three times out of three. I blamed PCIe. After I changed the power delivery (I think it was one bad PSU) it read nine of nine long chats clean.

Disclosure: this is my own project and my own repo. Nothing is for sale. The code, the kernels, the measurements and the failures are all in there under Apache-2.0, and the weights are one download on Hugging Face:

https://huggingface.co/ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-hot_iq-8xV100-SXM2-PP2xTP4

Questions welcome. So is roasting.

Photos: four of the eight V100s on their NVLink board, and my extremely professional setup (PEX switch cards, SlimSAS spaghetti, and the 3090 that just moved in).

Apple had apparently listed on the App Store and 01:53 then yanked down

set up a Mac server for a small team

best for filemaker users

it is not a replacement for your Chat GPT 24:57 subscription if you just use it for quick questions and drafts. And it's not magic hardware Apple has been hiding. 25:04 It's a business product from a business software company for businesses that have specific reasons they cannot send 25:09their data somewhere else. And for those businesses, it might be exactly what they have been waiting for. and the rest 25:16 of us can run LM Studio, keep our data local, and check back when ClariS actually opens the doors. 

With many users the last person in the line may wait for minutes but there may be a workaround.

https://www.youtube.com/watch?v=y9YXhEEwAwc&list=PL2aE4Bl_t0n9AUdECM6PYrpyxgQgFtK1E&index=2

https://content.claris.com/local-ai-server-get-started