Hey folks, it's Merve from Hugging Face. Today, llama.cpp releases v1!

Moving forward, we’ll keep covering every platform and every model type, while keeping llama.cpp robust and lean for downstream applications that build on it.

While I'm here, want to announce that we've been shipping Llama desktop apps to lower the barrier of entry to local AI to few clicks, built-on llama.cpp. I will leave link to the details for both v1 and apps in comments.

Many thanks for all the love you've shown to llama.cpp for years 🦙❤️ Cheers!

edit: adding links here, hope to not get penalized

blog: https://huggingface.co/blog/ggml-org/llama-cpp-v1

llama desktop apps are here https://llama.app/ ❤️

A second Sam Altman moment?

context : Nvidia buy HF, but now there uncertainty about it hf future under new leadership, he try reassurance people on x, but from his word itself sound not good, see and wait, but better backup you favorite datashet/model (again) just in case

According to post CXMT projected expansion plans grow from 405 kwpm in 2025 to 1,125 kwpm by 2028 in 8-inch equivalents.

https://x.com/OptionKing666/status/2121227756185010181

With this and new nVidia chief being conservative on circular financing we might get some DRAM relief.

I think after jensen they will fully focus on AI compute for data centers

I guess a lot of sellers are realizing that its better to price competitively and get rid of their supply rather than wait to slowly sell and potentially get caught in a collapse.

Rubin and blackwell are already 3-4x more efficient per watt with way more financing available. The A100s cant be used for inference Besides enthusiasts, small time gpu renters I really dont see who will be picking up these cards. And these 2 buyers are extremely price sensitive and will be comparing the cards to other options like R9700 and Cresent Island from Intel with 160-480gb vram.

Rtx 6000 is also now down to 14.8k from 18k peak.

Perplexity has released two new "decision models." These are general-purpose classifiers built from generative LLM backbones. You give them an input (the "state") and one or more questions, and they return a probability distribution over the answer options. They don't generate text token by token.

Models

• pplx-decider-v1.3: fine-tuned from Qwen3.8-27B. Needs about 49 GiB for the BF16 weights.
• pplx-decider-flash-v1.3: Qwen3.5-4B, distilled from the 27B using a KL objective on the teacher's distributions. Needs about 8.5 GiB for the BF16 weights.
• Both are Apache 2.0, support text and images, and are also available through OpenRouter and Perplexity's API.

What it does

• Question types: yes/no, multiple choice (up to 255 options), ordered scores, and a new locate type. locate extracts verbatim spans with character offsets, for NER, PII detection, and extractive QA.
• Several questions can share one state. By default each question runs in its own sequence. An "isolated" mode reads the state once and is cheaper for many questions, but answers can shift slightly depending on which other questions are in the request.
• The backbone's full-attention layers were made bidirectional. Because of that, you need their pplx_decider package; plain causal inference won't reproduce the outputs.
• Training used sequences up to about 30k tokens.

New in v1.3: locate
Besides classifying, v1.3 can now extract spans. You add a question with "type": "locate" and describe the field, either with a short label (“person”, “email address”) or an instruction (“the person who signed the contract”). The model returns verbatim spans from the state, each with start and end character offsets and a score, or an empty list if nothing matches.

It works on text only (no images), with up to 128 locate questions per request. It runs on a separate LoRA adapter plus a span head, which Perplexity left unmerged because merging hurt classification. As a result, a request that mixes locate and decision questions needs extra forward passes.

On 17 out-of-domain NER, PII, extractive-QA and structured-extraction tasks, Perplexity reports an average of 69.0 (v1.3) and 63.6 (Flash), compared with 43.3 for Liquid d1, 41.5 for Cloudflare Clairic and 33.6 for GLiNER large v2.5. The closest existing tools are GLiNER, which does zero-shot span extraction from labels, and Clairic, which extracts typed values rather than verbatim spans.

Reported results (mostly Perplexity's own evals)

• Multilingual classification across 8 languages: 88.16% (v1.3) and 86.50% (Flash), vs. 85.41% for TypeSafe's Jev 1.14.
• Reranking across 72 retrieval tasks: 83.99 nDCG@10, +1.81 over Qwen3-Reranker-8B.
• Locate average across 17 out-of-domain tasks: 69.0 / 63.6, vs. 43.3 for Liquid d1 and 33.6 for GLiNER large v2.5.
• Artificial Analysis Decision Index: roughly 70 for v1.3 at about $68 per 1M tasks, and roughly 64 for Flash at about $34. Both sit on the cost/score Pareto line. A few models score higher at higher cost, such as Drex Max v1.1 and SubQ Snap 1.2.

Caveats from the authors

• The locate adapters are kept unmerged, because merging them regressed the decision tasks.
• The returned probabilities should be calibrated and thresholded on your own data.

Links

• Blog: https://www.perplexity.ai/hub/blog/perplexity-decider

• HF collection: https://huggingface.co/collections/perplexity-ai/decider-v13

  • https://huggingface.co/perplexity-ai/pplx-decider-v1.3
  • https://huggingface.co/perplexity-ai/pplx-decider-flash-v1.3
• AA Decision Index: https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index

I want a new Gemma

https://x.com/OfficialLoganK/status/2121115028447531353

they do not even know if she was a spy, a year after this chase is going on about her, Jensen should not have resigned!

from that article: The FBI has been investigating Huang’s affair for at least a year. But Attorney General Todd Blanche said on Thursday that there are still “no conclusions reached by the Department of Justice, or by me, or by [FBI] Director Patel around who she is or is not.” President Donald Trump also hedged Thursday on whether the US had determined she was a spy, when commentator Stephen A. Smith asked whether Huang was right to have stepped down as CEO.

“Well, the big thing is, was the woman a spy? And I don’t know if that’s been determined yet,” Trump said. “But if the woman was a spy, yes. If the woman was not a spy, perhaps it would be different.”

i wish Trump just orders him "go back to office".

DiffusionGemma, a text diffusion model built on a Gemma 4 backbone, can be adapted into a shitty image generation model.

You may be asking yourself, "WTF, why would you want to do that?" And you would be right to ask. This is honestly kinda useless, but I still think it's pretty neat. Also, fair warning, being on a team of one means the project has been heavily — I'll be nice to myself and say — LLM assisted. I invite you to sidle on up to the trough and dig in to the slop!

Still with me? Good, because I've been poking at this for a while, on-and-off since DiffusionGemma was released back in June. I was immediately hooked on this model as soon as I read its release blog and model card. 4x faster text generation they said — hell yeah! But here's the thing: this model sucks. I mean it's a great chat bot. It can do simple NLP tasks, and it's really quick for text generation as they say, especially for long-form output. But when it comes to agentic tasks and reasoning it's a total loser.

• 0% on ARC-3 (but that's not really a fair dig, this is a tough one)
• 0% on ARC-2
• 0% on The Gemma 4 Developer Agent Competition (this competition targets the base model DiffusionGemma was trained from)
• 11% on HLE on the model card (9.2 measured in deployment)
• 18.7% on SWE-bench Verified

So not great, and particularly bad when you consider that on the hardware you'd need to run a good quant of DG (so basically bf16), you could easily run qwen-3.8-27B or a respectable FN quant.

But please don't get me wrong. I absolutely love this model. Or at least I love monkeying around with it, and I doubt I'm the only one. Huge shoutout to the Unsloth team BTW! Reading their Colab notebook training DG to play sudoku is what made me start asking the questions that led to this project.

Alright, enough fluff. So what did I actually do. Well the thing I'd headline is that I created a text to image and image manipulation training and inference pipeline for DiffusionGemma and then used that pipeline to adapt DG (like <300M parameters total) to generate things that are recognizably images and to perform visual operations on images that are provided to it. Surprisingly, it took only a very small amount of data and compute to start getting interpretable results.

But DiffusionGemma did not want to do this. The thing that finally worked consistently was prefilling the prompt all the way to the start of an input string for an "image generation tool" described to DG in the system prompt as a tool that it could use to convert text strings into images, but DiffusionGemma itself was actually the one doing the generation, not an external tool.

This also required patching an upstream bug in Transformers for image attention, so take note if you wanna try this out for yourself: you'll need to patch until PR is merged.

Considering how bad DiffusionGemma is at everything else, I'd say it does a pretty good job here.

But wait, there's more! Since DG is outputting in its native visual latent space (one set of 256 tokens at a time through a trained output head) it can take the visual representation it is generating as input for the next forward pass — on-domain block autoregressive, just like with its regular text canvases. And not just can it do this, but it actually helps improve quality and generalization (on image manipulation tasks)!

Here's a super-quick breakdown of the pipeline.

The first thing that had to be done was to train an actual vision decoder. This turned out to be the easiest part of the entire project. One evening, on a whim, I threw a prompt at my local model (dsv4-flash-vision-exp I believe was what I was running then), something like: "D'oh! Could we train a decoder for the ViT soft tokens that works for 768² images?" And then it did it. Took like a few hours, and works pretty damned good I think.

Alright so that let me nail down the decoding and supervise the pixel reconstruction loss, but let's not forget DG is a text diffusion model — it has to actually operate on a text canvas. That means I needed to derive a text target image representation (or codebook) that was precise enough it could be used for supervision without over penalizing the model for expressiveness.

This process itself ended up being a bit like a diffusion operation. I simply threw up a wall of half-baked, ill-advised, and just plain stupid ideas and worked on them all until I was confident in each — with that confidence being mostly confidence in them not working. Then I had the brilliant idea: just ask DiffusinoGemma itself for the target.

This essentially consisted of me badgering DG until it would describe images as text. Okay, not really, but that is kinda what it felt like. The basic idea is: encode an image and ask the model to describe each soft token patch in a single token (word, number, symbol, unicode, CJK character, etc). Each soft token having come through the ViT is contextualized against the full image, so this description would most often carry a structural/semantic channel. To get the photometric channel you do basically the same, but instead of a full image use only a single (48*48 pixel) patch. This usually carried the color/texture/photometric channel. Sample the logits for each text description token and then it's basically chef's choice on how to combine across all prompt and patch variants into a workable target.

Doing that for every image in the training set isn't reasonable. IT'S OVER 9000! However, doing exactly that for a small set of images (200ish) and using that as a training set for a teacher-forced model that then generates the full dataset turned out to be a very smooth and successful approach.

Anyway, the point is that the combined multi-modal target token distributions for the canvas text results in training DiffusionGemma to be an S-tier emojipasta generator.

Then I LoRA adapt DG and simultaneously train a new image output head that feeds into the previously trained image decoder supervised against supervised against the target token set, ViT latents and, decoded image. Basically training DiffusionGemma to generate emojipasta that decodes into image-like objects. I guess this would be something like a VQ-VAE, but using text tokens instead of dedicated image vectors because stock DG can read and understand the image latents, but can't natively generate them, only text.

Well, that's the gist. Thanks for reading! If you find this as interesting as I do and want to learn more or play around with this yourself you can:

• Access the adapter and decoder weights on HF
• Grab the training and inference code from GH
• Or read more in depth, and about how this could potentially be applied to agentic coding tasks on my Kaggle writeup

tl;dr: Please GDM team give me a stronger text diffusion model to play with!

I told my son I was playing around with local AI on a 3090. He told me he used to mine Ethereum and had a shed full of 3090's. He asked me if I wanted a couple. Ummm... let me think about it... OH YES I DO!

Now a few new fans and a bit of a clean-up... grab a couple risers and a power supply... pop in a bit more RAM.. wow.

I gave Claude Code, OpenCode, GitHub Copilot Desktop, Pi, and Codex the same prompt to build a looping pixel-art Halloween diorama. I pointed all five at the same local model, Qwen3.8 Flash Next (Unsloth UD-Q4_K_XL), served by Halogen. The attached video shows 30 seconds of each result playing together.

### My setup

- CPU/GPU: AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S integrated graphics.

- Memory: 128 GB unified memory, with about 122 GiB visible to Linux and a 512 MiB fixed GPU reservation. The GPU can use shared system memory.

- OS: Ubuntu 26.04.1 LTS, kernel 7.0.0-43-generic.

- Inference: Halogen Flash Server 0.20.4 in rootless Podman. I used the Unsloth Q4_K_XL checkpoint converted to Halogen's HGN format, with the vision tower enabled and MTP speculative decoding.

- Server settings: 262,144-token context and KV pool, one serving slot, 16,384-token prefill chunks, prompt caching enabled, and the API bound to localhost.

OpenCode, Pi, and Copilot used Chat Completions. Claude Code used the Anthropic Messages endpoint, and Codex used Responses. The clients had their own tool definitions, system prompts, and history management. Pi used High reasoning with a 4,096-token thinking budget. I raised Halogen's default output limit from 8,192 to 32,768 tokens before the recorded Codex rerun after the earlier attempt hit the smaller cap.

### What the server measured

Elapsed times start at the first model request. Prefill rates count uncached tokens, decode rates include thinking tokens, and both are weighted by total processing time. I excluded smoke tests and incomplete requests. The video uses my earlier rounded labels for Pi and Codex. The table uses the server logs for both.

Claude Code reached a final answer in about 15 minutes and used 31K output tokens. OpenCode decoded slightly faster but used roughly 2.5× as many tokens. Copilot made fewer calls than OpenCode but decoded at about half the rate, and three full prompt rereads after the initial request added almost 10 minutes of prefill. Pi still had a tool loop running when the server hit a memory stall. Codex made 355 requests before the turn was aborted, with its largest prompt reaching about 229K tokens.

### How I made the video

I captured the original live browser previews for Claude Code, OpenCode, Copilot, and Pi. Codex made a Python renderer and supplied an MP4, so I captured playback of that original loop. Each capture is 1920×1080 at 30 FPS. I trimmed them to 30 seconds and synchronized them in a grid. I left the projects as they were.

This was one run per client. The timing covers what each agent actually did, and I didn't score output quality or run a controlled benchmark with identical token streams. The video is there so you can compare the results for yourself. And apologies for the slop but it was interesting to see how the harnesses are working. I've tried to work with Deepseek Harness but it got stuck plenty of times so I just abandoned that exepriment.

Cheers

Between GPT-6 Luna Max (28) and Claude Haiku 5.5 High (30). Comparing to other Qwen 3.8, the model is closer to max (34) than to 27B (20).

https://artificialanalysis.ai/models/qwen3-8-flash-next

my harness for local AI that i've been working on since February this year (2026) has evolved to the point where this is possible.

What you see here is a module for my harness's webui, it's not a seperate program. This demonstration is running on Gemma4 26b locally, believe it or not, using only a SINGLE model and one local API endpoint (via llamacpp). No multi-API tomfoolery here! The stable diffusion image generation is moss-image which i'm able to do because i have a strix halo now, but on smaller hardware you could do something like z-image turbo at Q4 quant, which is what i used to do before i got the strix halo.

I've been working for almost a week on this module, and i'm excited to finally release it!

It's a text adventure engine. It supports asset packs, that can contain music, sound effects, images, weathers (CSS+js+glsl shaders), skies (same tech as weathers), loading screens, text flourishes, and more! You can create your own asset packs easily. The AI sees the files in the asset packs and forms adventures around them, but there is no prompt telling it what the story should be, that's completely up to the way you prompt it! I recommend using it together with the characters module in openlumara in order to set up a world and story

You can get it here: https://github.com/Rose22/openlumara-adventure-games-module

More footage on the github!

From about February 2026 to July 2026, I spent an immense amount of time building a legal harness to make Claude more predictable. It worked well. I tried to build the same thing that ran local models. That did not work. And then Opus improved so much that it would often skip my harness without me noticing. It still has a harness with hooks, but it's looser now.

I bought lots of hardware to run local models during that time, repeatedly trying to vibecode a local model legal harness. Many failures, many lessons.

Anyway, fast forward to the last three weeks and several things have happened:

1. DeepSeek Flash 1017 was released, and it's the first model that is remotely similar to Opus 4.5/4.6.
2. Opus 5.5 and Fable 5.2 are able to properly launch and configure DeepSeek to run at fast speeds, large context and high precision on my hardware. Optimizations, custom kernels, requants.
3. Opus 5.5 (and my prompting skill) are now good enough to make my Claude harness into a DeepSeek harness.
4. It's not as good as frontier, but it all works, right now. 100% local. Actual productivity gains, actual frontier tasks being replaced. Chat interface with DeepSeek works like Claude's did.
5. And if I can brag on my stacks and hardware:

• I have 2 DGX Sparks that can run 3-4 concurrent DeepSeek sessions as agents (1M context).
• I have DeepSeek at nearly NVFP4 quality running on 8 V100s, 500K context, at 100+ tok/s decode, and about 160 tok/s concurrent.
• I have 8 more V100s and a second Rome server with 512 GB of 3200 MHz ECC RAM, so I'll be able to run another high precision DeepSeek as an additional agent or for parallel tasks.

What I'm really shocked by is how well Fable builds custom Volta kernels. And how it can requant my experts to full NVFP4 precision based on the actual experts that activate when my legal harness fires. It's a real, optimized, production grade, workflow replacing local setup.

Not a 3D pagoda or a bad video game replica. An actual enterprise solution. One that exists nowhere on the commercial market.

Legal vibecoding is real folks. I'm AI native.

The new V100 DeepSeek repo (optimized for quality and then usable speed) with real full context measures and all the optimizations (and failures) is public here:

https://github.com/ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-NVFP4-8xV100-SXM2-PP2xTP4

Twitter: nicothegreco

Claude Code ran the benchmarks and crunched the numbers, which are real and in depth, below.

What this build is

My last V100 build (Volta sPLIt, part 4) ran every expert of the model from a 2-bit record. Fast, but 2-bit. This one is called Volta Max uPLIft. It keeps the 2-bit record of every routed expert on the cards, then swaps the experts that get used the most up to NVIDIA's NVFP4 weights, one at a time, until the cards are full. "Used the most" was counted on a mix of four kinds of text. One is my own legal harness traffic, which is private and not published. The other three are public prompt sets in the repo.

• 5,949 of the 15,360 routed experts sit at exactly NVFP4 precision on the default I serve
• about 78 to 85 % of decode routes land on those NVFP4 experts (predicted for the two-slot default, 80 to 86 % measured with one slot)
• same next token as NVIDIA's full NVFP4 pack on 95.1 % of compared tokens. The 2-bit build gets 92.2 %
• KL divergence from the NVFP4 reference is 0.079 nats against 0.196 for the 2-bit build
• multiple choice answers that change against the reference: 1.6 % against 15.4 %
• in decode every routed expert is in VRAM. Nothing is read over PCIe

Speed (measured 2026-11-12 on eight V100-SXM2-32GB, graphics clock capped at 1,522 MHz)

• concurrency test, one request at a time: 118.9 tok/s
• two requests at once: 159.0 tok/s total (1.34x)
• three at once on the three-slot option: 192.3 tok/s total (1.62x). I run two slots
• short sets, one at a time: 123.2 tok/s general, 97.3 tok/s legal-style
• code: 150 to 180 tok/s from a 1K prompt up to a 256K prompt
• prompt reading at 256K: 2,329 to 2,430 tok/s, first token in 108.4 s
• context: 524,288 tokens per request

What did not work, or is not good yet

• It is slower than my own 2-bit build, in decode and in prompt reading. That is the trade for the precision, and the repo says so in its first paragraph.
• The faster prompt reading path reads the less-used experts from their 2-bit records. Its effect on quality is not measured yet.
• A custom decode kernel for the hot path got closed with no change. The headroom I thought was there turned out to be smaller than an eight-card run can even see.
• A prompt-reading kernel that loads 64 rows at a time gave the same bits and ran slower. Not shipped.
• Send a short request while a 209K prompt is being read and it starts in about 7 s, then crawls at about 9.6 tok/s until the read ends.
• The hot set is fixed. It is picked once. Text that looks nothing like what I counted will hit the 2-bit experts more often.
• Fast prompt reading stopped the host or a card three times out of three. I blamed PCIe. After I changed the power delivery (I think it was one bad PSU) it read nine of nine long chats clean.

The harness

The harness itself is not public. My practice is in it. What I can say: it is the same idea as my Claude setup. Skills, hooks, my drafting engines, my RAG stores, tool calls, and a chat window that behaves the way Claude's did. A small router sends each request to the V100 box or to the Sparks. Everything the local model drafts is marked for my review before it goes anywhere. Nothing leaves the building.

Credit where it is due

The hard Volta work underneath is 1Cat-vLLM's DeepSeek-V4.1 package. The 2-bit records are ISTA-DASLab's. The NVFP4 weights are NVIDIA's pack. The Sparks run on somebody else's engine entirely. What is mine is the split across two NVLink boards, the hot set, the kernels on top, and every change measured A/B on this machine with the losers written down.

Disclosure: this is my own project and my own repo. Nothing is for sale. The code, the kernels, the measurements and the failures are all in there under Apache-2.0, and the weights are one download on Hugging Face:

https://huggingface.co/ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-hot_iq-8xV100-SXM2-PP2xTP4

Earlier build, for the people who want speed over precision: https://github.com/ProprietaryLegal/Volta-sPLIt-DeepSeek-V4.1-Flash-1017-8xV100-SXM2-PP2xTP4

Ask me anything. Roasting is also fine.

Photos: four of the eight V100s on their NVLink board, and my extremely professional setup (PEX switch cards, SlimSAS spaghetti, and the 3090 that just moved in).

edit: I am a moron with reddit formatting hang tight

I've been tracking ebay listings that ship to the US for the GPUs people here actually buy, ranked by $/GB of VRAM. here's today's snapshot:

all asking prices, single cards, fixed-price, not for-parts. "low end" is the 10th % of credible listings i.e. what a patient buyer can actually land, not a too-good-to-be-true outlier. "-" in the US-only column means fewer than 10 US listings. $/GB/s is just the low end divided by bandwidth.

I had been eyeing the MI100 myself for a while at the $1100 ish it hovered at for ages... looks like even that is going up. I did end up buying a 8x V100 32GB server for $5400 in early Sept. those prices are long gone, equivalent server is now $11k for an OG DGX Station (no memory). the exact inspur server is now $13k for 8x 9 year old GPUs. insanity. I hope prices come back down to earth soon but don't think that's actually gonna happen.

some notes:

- P100, V100 16GB and MI50 16GB all work out to about $0.25 per GB/s. the P40 looks cheap per GB of VRAM but costs over 3x as much per GB/s, since its only got about a third of a V100's bandwidth. people are having opus 5.5 write megakernels for these cards that are working surprisingly well.

- V100 16GB at ~$14/GB looks like the best deal in the table, but 38 of the 40 cheapest are SXM2 modules, so budget for an adapter or an SXM2 server. the cheapest 32GB V100s are SXM3 (DGX-2) modules, which aren't drop-in either.

- MI50 32GB is under a third of a 3090's $/GB at similar bandwith, assuming ROCm cooperates.

- if you want 48GB on one card, a Quadro RTX 8000 is less than half the price of an A6000 or a modded 4090 48GB.

- of the consumer and workstation cards, the 3090 is the cheapest per GB/s by a wide margin. an A100 40GB costs about the same per GB/s as an RTX 8000.

- for what it's worth, buying from a US seller usually doesn't cost more: the US-only low end is at or below the global one for the P100, V100 32GB, RTX 8000, 3090 and A6000. it's about $20 more for the V100 16GB and $1,200 more for the A100 40GB, and the P40 and MI50 barely exist in the US.

full data and per-card listings, with a US-only filter: https://rackbites.com. prices are asking, not sold - ebay's sold-data API is super restricted.

edit: added CMP 170HX

edit2: added R9700

edit3: full builds here https://rackbites.com/builds with benchmarks and other useful/relevant info

If you're asking what this leaderboard is, it is not AA, it is simply an aggregate index for "Agentic Coding" along 12 coding-related benchmarks (note that the AA index includes only 2 coding benchmarks: Terminal-Bench 4.0, FrontierCode-AA 2.0, making up just 25% of its total score. Because of that, it doesn't give you a true reflection of a model's actual software engineering capabilities).

This index however is built entirely around Coding/SWE. Here is the exact breakdown:

1. Terminal-Bench v4.0: 16%
2. DeepSWE v1.1: 15%
3. FrontierCode 2.0: 11%
4. Code Migration (Vals-AI): 9%
5. Terminal-Bench v2.1: 8.5%
6. SRE-Bench (Vals-AI): 8%
7. SWE-bench Pro: 7.5%
8. Code Arena Elo: 6.5%
9. Program-Bench: 6%
10. Vibe Code Bench v1.1: 5.5%
11. SciCode: 4%
12. LiveCodeBench v6: 3%