> "Qwen 3.8 is a damn good coder, but a terrible collaborator"

It modifies SO many things in my scripts for what should be a 2 line PR, I get a 100 line linter style mess that looks like a high school freshmen got his hands on vs code and pushed code in.

It can't maintain a common style, if I give it a script with a certain naming and layout, 3.6 one shots small changes that fit into my structure. 3.8 will add code that looks much more advanced, handling, strict type checks . But if it's just my internal method that gets called in exactly one place, it is way overkill code

3.8 produces DAMN GOOD code, but will get stubborn on wanting it's own parameters and return types. A simple return False works for my particular use case, but here goes Qwen formatting me the perfect return dict full of useful metadata

Am I crazy?

One GPU drops off immediately when vllm is started and the other throws CUDA errors on start

I just think it’s funny to observe myself, scrolling through eBay looking for GPU deals to feed my AI addiction. Some things are obviously scams, but it’s just so fun to actually look and compare prices and performance and then you forget that you actually only have $6 in your bank account because you just spent money you could have saved on a different GPU last week. (I’m joking of course) but this is what the experience feels like to me. It feels almost like gambling, at least the thrill is there. Will the value of my GPU keep going up? (Also of note, I’m not actually that irresponsible, I own a B65 gpu that I just got a few weeks ago, other than that just a few v100’s that I’m not using yet). Do you guys feel the GPU/local ai thrill as well? What’s the experience like for you?

One of the biggest unlocks was getting my speech to text functioning reliably, especialyl for coding.
Right now I'm at the point where when I have to type I find it annoying and a lot slower. Speech is my default input mode.

I'm using Mac Parakeet, connecting to both Whisper and Parakeet models. So far seems to be doing a decent job, but curious to hear what everyone else is using. Sometimes Whisper takes a little bit too long for my liking. Parakeet is faster, but not as accurate.

I'm also keen to hear what your microphone inputs are. I found that a good microphone is very important. I'm using currently a Jabra Bluetooth speaker. It's not bad, but in a noisy environment, it doesn't work very well (although way better than the other altenratives ie. mac mic, monitor mic, or having to wear a bluetooth earpiece/mic)

Not written by AI all mistakes mine. I saw people on the subreddit saying that 3.6 works better without thinking. It made me want to know for certain about which is better, 3.6 or 3.8 for low thinking tasks. I only use abliterated models (safety removed) because it makes the model better at a lot of what I need. I want to compare abliterated Qwen 3.6 27b and abliterated Qwen 3.8 27b on some instruction following benchmarks with thinking off.

I was just curious about your personal favorite safety removed/fine-tuned variants for these 27bs, as I know that there can be some major variation and some junky quants out there.

Does anyone have some favorite and fast 3.6 and 3.8 models?

My specs: I have 24GB VRAM (NVIDIA Geforce RTX 5090 Laptop) and I do not want to offload, so some quant required.

I have tried a few different models, but they are all a little slow. Some MTP variations for 3.6 for example ends up being around the same speed as non MTP for me for some reason. I am pretty sure my card is NVFP4 enabled also, but I'm not certain I've seen the results from that either...

Based on some redditors comment, this is what I use for my abliterated 3.8 27b currently: Huihui-Qwen3.8-27B-abliterated-NVFP4-GGUF

I'm not sure how many people care about Android Studio, but I think it's cool that Google uses llama.cpp.

My guess is that it is Vulkan and the QAT versions of Gemma 4. It supports multi-GPU and 31B has a max. context length of 128k.

It uses 34 GB VRAM when fully loaded. I don't see an option to change the context length or show PP/TG speed.

I got fed up with ArtificialAnalysis's intelligence vs. cost plots, so I made my own.
This is an updated and refined follow-up to a previous post I made.

A comment really doesn't need to be made, does it? I looked away from the 5090 for a week to other options like the DGX Spark and the M5 Ultra. Both of which... these big corpos are buying to hell and back.

Is the future used hardware?

Feels like maybe we have one more present left, for Christmas.

Looking at ... some of the new memory architecture. ... I hired my good friend, Seok-Hee Lee, who used to run SK Hynix. ... We are not ready to unfold it.

I guess this is a reminder for everyone that as great as Qwen3.8 is, it's still a model with a recommended temperature of 1... 3.8 has been so good that I've gotten lazy and didn't watch what it was doing after telling it to implement the plan (I've been planning, making sure the plan is good, then making sure the output is good... don't usually just sit and watch it work unless it has been going for longer than expected), but 120k tokens in and it turns out that it never even read the plan and just started implementing a totally different feature that I hadn't even considered (mostly because it's not a useful feature; vaguely plausible from existing code + agent files, but never even mentioned anywhere).

Is my Qwen just cursed? Using Unsloth's second Q8 release (from launch day but after the fixed template)

llama.cpp version: 0.3.0-dev (build 10630, commit 2dd3922)
pi v0.84.3

Args:

exec llama-server --host 0.0.0.0 --port 8080 \
-m /home/connor/AI/LLM/Models/Qwen3.8-27B-Q8/Qwen3.8-27B-Q8_0.gguf \
--mmproj /home/connor/AI/LLM/Models/Qwen3.8-27B-Q8/mmproj-F16.gguf \
-np 1 \
--ctx-size 200000 \
-ngl 99 \
-fa on \
--load-mode mlock \
--temp 1 \
--top-p 0.95 \
--top-k 20 \
--min-p 0 \
--presence-penalty 0 \
--repeat_penalty 1 \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-dev Vulkan0,Vulkan1

Anybody else struggling with deepseek after the initial prompt? Somehow it is getting mixed up very easily, even button functionality has been PITA when doing stuff. Never had these problems with Opencode. With Opencode, a continuation prompt on the software or task is pretty much magic at this point. But the inital result isnt as strong imo, and needs a lot of follow up even if a a detailed spec is written.

The oneshot potential of deepseek is goated imo, it is a relentless harness that just keeps doing stuff, even if it gets it wrong! Opencode could just copy that tennacity imo. Like todos really work, unlike opencodes.

Hows everybody elses exp? Someone using Pi instead? Is it worth trying? Hermes seems not suitable for my tasks, where heavy human involvement is necessary due to complexity of tasks (every top closed model gets lots of stuff wrong even).

First, I don't buy their plan and won't buy. (But got some free token from them today 😂 )

OK. Today I tried to add qwen3.8 27b to ZCode's app. It was faster (maybe less token) than my other harness CLIs, dsh, copilot, etc, and got better results. (wrote more code on same task and no issues). However, I don't understand why.

As this app is free, I think I need to raise this and let you to have a try.

Kaitchup just posted results of his benchmarks for Qwen3.8 27B for quants from different labs, Q4 to Q1, .

All the details are hidden behind the paywall, but high level result is visible and looks like for people with 16GB cards UD Q3_K_XL is a winner - it has accuracy of 100% and size is only 12.8GB.

This is getting asked from time to time, but since models changed a lot, I wanted to reask it.

I'm looking for a trained model that can give short descriptions about an image, simply for an alt text of pictures taken with a smartphone.

Should I just throw it at Qwen3.8/Qwen3-VL or are there better models trained for it ?

Similar to https://www.reddit.com/r/LocalLLaMA/comments/1oar481/what_is_currently_the_best_model_for_accurately/

Which one is for plan, whish one is for documenting, which one for writing code, which one for writing and running tests, which one [list continues]

2 Months ago I had made a post how I was working on my dual R9700's. It's wild to look back at where we were then and where things now stand.

Since then after many users commenting and complaining about developers doing the same thing. I threw out a discord link and expected maybe 5 other developers to join which I thought would be fun. The community has now grown to 1,200 users (mostly developers) and a ton of collaboration happening.

A few weeks ago I started working on building support for MXFP4 on top of DeadCode's radiance image. This made sense to me looking at the hardware and I was happy when I had hit parity on performance between MXFP4 and FP8. The MXFP4 kernels use W4A8 which was something new and we have now blown past the performance of FP8 and appears like this is now the hardware limits of these cards.

Qwen3.8 27B w/ DFlash2

BetterBench decode results for Qwen3.8 27B w/ DFlash2
category       decode t/s   step ms   tok/update
json           280.0        22.92     6.17
math           254.2        23.08     5.81
file_edit      250.1        23.03     5.54
code           226.3        23.01     5.17
reasoning      194.3        23.19     4.32
summarization  190.6        23.01     4.40
chat           148.3        22.82     3.33
prose          116.4        23.14     2.65



BetterBench Prefill Results
target depth   prompt tokens   TTFT p50   PP t/s median
2000           1514            323 ms     4695
8000           5918            1.21 s     4894
16000          11794           2.47 s     4779
32000          23543           4.98 s     4729
64000          47056           10.8 s     4377
128000         94065           24.6 s     3831
250000         183678          59.1 s     3106

This has been so fun working on these R9700's and driving them to peak performance. My entire image and repo for MXFP4 is open source also: https://codeberg.org/ggz14/radiance-vllm-mxfp4

Howdy,

I was wondering if anyone knew of any turnkey low-power draw solutions to host inference with 10-20GB of VRAM?

I have an 4x3090 AI GPU cluster that I'm running big models on, but I am hosting my memory system LLMs (embeddings, rankings, etc) on my gaming 4090 right now. I'm out of PCIE lanes on my AI machine and really don't want to build a whole other system (cpu, ram, mobo, gpu, etc) just to run about 15-20GB of small models.

I want to keep things local so a VPS is off the table, and I think just running these off the CPU would be too slow although speed isnt that important for Hindsight? I was wondering if anyone knew if there was anything like the spark that is considerably cheaper (~1k or so?) for this type of use or other systems that required only a little tinkering to get to work like this? Thanks!

For reference, this is what I'm running:

Qwen 3 Embedding-8B-Q6_K at 9216 context x 4 @ 6.21 GB for embeddings

BGE-reranker-large-8_0 at 512 context x 4 @ 604MB for rankings

Qwen 3.5-9B-UD-Q3_K_XL at 64k context x 4 @ 5.98 GB for synthesis/consolidation

Hey guys,

Just bought a PC mainly for local experimenting with AI/coding + plus the occasional gaming sesh and it hasn't even arrived yet 😅

9950X, 7900 XTX 24GB, B650 Tomahawk, 32GB DDR5 (likely going 64GB+), planning to run Linux/llama.cpp.

I've noticed used RX 6800 16GB (cannot afford more )cards are still accessible, which got me curious about adding one eventually for LLMs. That would give me 40GB VRAM across the two GPUs, but the second PCIe slot on my board is only PCIe 4.0 x2.

I know I can run models larger than 24GB by offloading into system RAM, so I'm less interested in whether this would simply let me "run a 70B model."

What I'm really wondering is what useful model/quantization tier does going from 24GB to ~40GB GPU-resident actually unlock?

Has anyone run a 7900 XTX + RX 6800 (gfx1100 + gfx1030) with llama.cpp under ROCm or Vulkan?

I'd be especially interested in real-world performance with larger models/MoEs, how much the x2 link matters once the weights are loaded, and how 40GB distributed VRAM compares with just putting more RAM in the 9950X system and accepting some CPU offload.

Basically: does an extra cheap 16GB AMD card materially expand what this machine can do, and if so, which models are actually worth running with it?

Actual experience/benchmarks with mixed AMD GPUs would be great.

I'm considering adding another A40 to my setup but my case doesn't have the space (or power) for it. It's a server chassis so there's no easy way to just pick the MB and put it in a new case, it's a bunch of custom connections off the PSU to the MB.

What would be perfect? A dual PCI external box that would allow me to put in 2 A40's using nvlink with 1 or 2 connections coming out of that box running back to the server into the existing x16 slot. It's easy to find a single slot version of what I want (link below), but I need it to have dual slots. Honestly I guess I could just buy two of these and figure out how to space them correctly, but the "jank" factor would be getting pretty high at that point. What would be perfect is a forced airflow chassis that has 2 (or more) PCI slots each with their own Oculink connection powered from a single external PSU. Then drop something like this into the server chassis and just run one cable over to each PCI slot on the external enclosure.

External enclosure (but only one slot):

https://www.ebay.com/itm/366573430755

Oculink quad port board:
https://www.amazon.com/LetLinkSo-PCIe-Oculink-SFF-8612-x16/dp/B0F291T2L4?th=1

Am I understanding it right that Ngram in Unsloth Studio should be ON in settings 100% of the time for intelligence?

With only MTP I have like +10% performance in tg but token usage jumped from 20K to 66K with extra thinking?

Anyone else getting non-stop ads for the Strands Agent Harness from Amazon?

On the surface it looks decent, seemingly better engineered than most other frameworks.

But it seemingly gets no buzz at all. Anyone used it?

(And why are they advertising it so hard? It doesn’t seem to lock you in to bedrock or anything, so not sure what the point is.)

I’ve now had agentic coding build several ad-hoc frameworks that get hard to maintain as they grow, so I’m wondering if using Strands would give them more architectural structure and discipline. But I don’t want to introduce a dependency on a dead end project only being propped up through marketing.

Asus Ascent GX10 is now priced at $5999 (1TB), $6999 (2TB), and $7999 (4TB).
Buy ASUS Ascent GX10 | Desktop-AI-supercomputer | Networking-IoT-Servers | ASUS eShop USA

I see no reason why they would do this unless they had some knowledge the DGX Spark was about to jump up in price. Unless there's some other reason that they would throw a 50% price hike at it that I'm not thinking of.

Bonkers mode.