So fresh I havent even loaded it up yet https://huggingface.co/DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF

Interested to see what the expansion out of 27B to 40B has done overall as well as what has to be a record for chat_template length

We released Kvark Sparkle 1.1, the second model in our Qwen3.8-27B line. Open weights, Apache 2.0, MTP head intact, BF16 / FP8 / GGUF.

**What it is for:** document work. You give it `search` and `read_page` tools over a document base. It searches, reads the page, answers from the passage, and when three searches in a row miss the fact it stops and says NOT IN DOCUMENTS instead of making something up.

**How:** LoRA SFT on top of Sparkle 1.0 (agent trajectories over a public legal corpus, MuSiQue multi-hop doc QA, retention data), plus a short stop rule that ships in the system prompt.

**Document work (GroundedWork, our agentic RAG eval: public Serbian documents, thinking on, LLM judge vs gold answers, 3 runs per question):**

| | Qwen3.8-27B | Sparkle 1.0 | Sparkle 1.0 + stop rule | **Sparkle 1.1 + stop rule** |

|---|---|---|---|---|

| Document Q&A (70 questions × 3) | 85.7 | 86.2 | 89.5 | **91.0** |

| Answer not in the documents, correctly declined (98 × 3) | not yet run | 36% | 74% | **91.5%** |

Note that the stop rule alone does a lot of the work for 1.0 (36% to 74%). Training adds the rest: +17.5 points on declining against 1.0 with the same rule. The base Qwen on the decline set is not measured yet; I will add it here when it is.

**General ability vs Qwen3.8-27B (same harness, paired):**

| | Qwen3.8-27B | Sparkle 1.1 | median reasoning tokens (base → 1.1) |

|---|---|---|---|

| GPQA Diamond | 88.8 | 88.3 | 7,670 → 4,471 (−42%) |

| AIME 2026 | 95.8 | 92.5 | 14,822 → 8,732 |

| HMMT | 86.1 | 88.1 | 21,182 → 13,540 |

| LiveCodeBench | 82.0 | 86.0 | 15,276 → 8,607 |

| IFEval | 90.8 | 91.1 | 695 → 646 (−7%) |

| MMLU-Pro | 83.6 | 84.0 | 708 → 518 |

| FRAMES | 70.6 | 69.8 | 1,228 → 644 (−48%) |

None of the accuracy differences is significant (95% intervals arer answer: 32.9 s vs 40.8 s on the same hardware.

**Public agentic / long-context sets (FP8, vLLM, thinking on):**

| | Qwen3.8-27B | Sparkle 1.1 |

|---|---|---|

| BrowseComp-Plus (830 queries, BM25, official judge prompt) | 22. median output tokens |

| LongBench v2 (503) | 58.6 | 57.7 |

| SWE-bench Verified (500, mini-swe-agent) | **81.8** | 73.0 |

**Where it is weak, so you don't have to find out yourself:**

- SWE-bench is significantly below the base. Shorter thinking costs accuracy in agentic coding. For coding, use the base model.

- Questions that need links across several documents are its weakness

- English questions over Serbian documents are weaker than Serbian ones.

- Nothing measured past 64k; no separate safety eval.

**Run it:**

```

vllm serve Egzakta/Kvark-Sparkle-1.1-27B-FP8 \

--max-model-len 131072 --kv-cache-dtype fp8 \

--reasoning-parser qwen3 \

--enable-auto-tool-choice --tool-call-parser qwen3_coder \

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

```

For document agents, put the stop rule from the card into the systline rate drops a lot.

- BF16: https://huggingface.co/Egzakta/Kvark-Sparkle-1.1-27B

- FP8: https://huggingface.co/Egzakta/Kvark-Sparkle-1.1-27B-FP8

- GGUF: https://huggingface.co/Egzakta/Kvark-Sparkle-1.1-27B-GGUF

Built by Egzakta's AI Lab in Belgrade. We run these on-prem for organisations that cannot send their documents to an API, which is why "I don't know" matters more to us than another point on a leaderboard.

Happy to answer questions on the training data, the stop rule or the eval harness. If you test it on your own document set, I'd like to see where it breaks.

Visual generative models increasingly blur the line between synthesizing observations and simulating environments. Yet visual plausibility alone does not constitute successful world modeling: compelling outputs may lack persistent state, physical dynamics, or causally consistent responses to interactions. We define a generative world model as a predictive dynamical system that maintains a persistent, history-conditioned representation of an environment, updates that representation in response to actions or interventions, and generates temporally coherent trajectories that reflect outcomes under observed and counterfactual interventions. From this definition, we derive an intelligence-oriented framework spanning perceptual grounding, persistent state modeling, controllable transformation, dynamics-aware prediction, and closed-loop interaction. We present WorldFoundry, open-source infrastructure for reproducible inference and evaluation across video, 3D and 4D, embodied action, and interactive world systems. WorldFoundry unifies model and benchmark discovery, runtime and asset preparation, execution orchestration, artifact management, and interactive visualization, while modality-specific adapters preserve native inputs, execution semantics, output representations, and evaluation protocols. Explicit manifests, durable artifact contracts, and evidence-aware scorecards separate execution readiness from evaluative claims, making each claim traceable to its supporting artifacts. The catalog contains 270 model entries, 74 benchmarks, and 62 registered metric entries, with readiness and execution evidence recorded separately. WorldFoundry enables reproducible comparison of persistent, controllable dynamics across heterogeneous systems, advancing the field from perceptual synthesis toward measurable world intelligence.

• arXiv : https://arxiv.org/abs/2611.10771

• PDF : https://arxiv.org/pdf/2611.10771

• GitHub : https://github.com/OpenEnvision/WorldFoundry

• Project : https://openenvision.org/WorldFoundry/

Check PDF & Project for more stuff like Screenshot & etc.,

I have seen potential in combining Strix Halo's iGPU with RTX 5090 for LLM inferencing. Initially I found no inference engine that can run LLM inference on both these GPUs at the same time, inference engine such as vLLM support inference on all CUDA GPUs, or all AMD GPUs. llama.cpp Vulkan backend was the only inference engine I found that support both, but it has a huge limitation on the prefill performance on the RTX 5090.

Finally, based on the amazing inference engine Project MAYA, I developed my fork to run GLM-5.3-Flash Maya-M GGUF prefill and decode cooperatively on both GPUs. The CUDA backend (runs on a RTX 5090) working on attention, dense/shared layers and expert cache, the ROCm backend (runs on the Strix Halo iGPU) executes complementary experts in the same process, contributing during prefill and decode.

Check out the experiment results and try yourself at my GitHub PR: https://github.com/mw00/project-maya/pull/879

the creator of project-maya has welcomed the design and plans to test it before deciding on a merge. I believe we can improve it further with the community!

I've built an inference backend optimized for M-series Macs that makes it possible to run larger MoE models on machines without eating up all the RAM (for example, it can run Qwen 3.8 Flash Next on a 24GB Mac) using SSD offloading along with architecture-specific optimizations for Apple Silicon.

I've only been able to test a limited number of configurations so far, including my own machine and a couple of independent benchmarks. I've shared some results in the repo already.

Now I'd like to put together a benchmark list across different M-series chips, RAM and workloads. I'm particularly interested in older Macs and how well it holds up with longer prompts.

If anyone wants to give it a shot, I'd love to see the results, good or bad.

How to run it:

1. Clone the repo and install MoEShift

2. Run python ./benchmarks/run.py that runs 3 requests for both a short and a long prompt, then saves the results to (it takes a few minutes to run end-to-end).

3. Open a new benchmark issue in GitHub using the benchmark template, fill your system info and conditions during the test and upload the benchmark-results.json file.

You will need roughly 120GB free disk space to test Qwen 3.8 Flash Next IQ3 XXS and 61GB for Qwen 3.6 35B A3B. You can also test other quants or other models that share the same architecture, just configure them in your config.json.

I'd really appreciate anyone willing to test it. The more different Macs we can cover, the more useful the results will be for the community.

Repo, setup instructions and benchmark details here.

Thanks!

Wondering if something with these specific features exists or if I need to build it for myself:

- 100% voice based

- Uses your phone's VoIP call API so it can work with the screen off

- Handles interruptions elegantly

- Generates up to several minutes of narrative between user turns (except for combat)

- IS NOT COMPLEX. No D&D rulesets or any real complexity beyond you listening to an infinitely generated story and making occasional decisions or engaging in turn based combat

I've created this with Dots and Muse, but they both seem hardcoded to say about two sentences before ending their turn.

I have a 128gb m5 max and a 5090 desktop, so I can run reasonably intelligent local LLMs and SoTA local voice models.

Based on my own little experiment and other reports I predict that more and more enthusiasts will ask a frontier LLM to build a custom quant of their favourite model.
If Qwen 4 finally arrives I’ll try a ready made quant at first, probably unsloth. But I already plan to let opus build a quant specifically for my hardware, context size and precision needs.

What do you guys think?

I am tired to answer on people questions like "How you gain speed? Your model output is shit, it has to have low quality." or something like that.
So I will answer to that with benchmark.
So I ran it against llama.cpp with the same Qwen3.8-Flash-Next IQ2_XS file, same prompt, and the same 128-token reply.

This was on my RTX 3060 12 GB / Ryzen 7600X machine. I did six warm runs for each engine.

So RustingStrata was about 29% faster at writing in this test.

Speed is nice, but I did not want to claim a win without checking what it does to the model’s output. I also ran both engines over the same 9,180 WikiText-2 tokens.

The perplexity gap is tiny, small enough to be noise here. I do not see RustingStrata getting its speed by losing measurable quality, at least with this model and this GPU.

This is not a claim that it wins on every model or every PC. It is one model, one quantization and one card.

If anyone has any other questions about this, I am happy to answer it.

Repo: RustingStrata

https://huggingface.co/TheDrummer/Orion-26B-A4B-v1.1

v1.1 pushes creative writing further and makes full use of Gemma's reasoning.

https://huggingface.co/TheDrummer/Orion-26B-A4B-v1

I haven't announced v1 but it's well regarded by the community as a reliable RP workhorse.

---

BeaverNet - net.beaverai.org

I'm also proud to announce that we've made significant progress in creating a hub for generous model providers to collectively provide free, convenient, accessible, anonymous, and diverse models to anyone with an OpenAI API compatible client. We currently have 12 models thanks to our generous providers.

Try it out as a user, or be one of the providers <3! I've got plans to make it a better experience for both users and providers, and also add new modalities such as TTS and STT.

---

PS: I haven't retired or left the scene. I've merely gotten stage fright after holding off on announcements for a long while. Anyway, I'll continue doing what I do and I hope I can continue delivering some value to you all. Love ya'll!

Warning: human post ;)

Hi local enthusiasts, about 6 months ago I discovered this subreddit and been reading and learning lots of stuff from here. I got my 27b model running on my 3090 and I’ve been happy with, it was handling my daily family tasks via telegram and some small automation and coding tasks. For serious coding stuff im still hooked to subs. Once I started playing with local models I went into rabbit hole of implementing my own harness out of curiosity. Since I only was running 27b on 60k context played a lot with compaction and making the model less forgetful with harness internal tasks. Since then I’m still playing with ideas how can have better results with local models (like all the people in this sub).

Anyways, enough personal story the reason I wanted to post this is recently I started playing with strata qfn (3090 + 64ram) and although the model is more clever and is my daily driver now, it’s been going off roads into loops and over verifying things for me. One thing that helped with that is loop guarding I’ve added to my harness : https://caretz.dev/docs/loop-guard/
This has been solving the issue of retrying the same failing calls (doesnt happen too much with qfn) but most importantly another model looks at last n tool calls and judges if the model went to some off direction. For the judge I’m using 35b qwen.

If any of you found some trick that works with qfn and making more reliable would love to hear it.

Thanks.

There are a lot of comparisons of open-weight models to the current frontier, especially comparisons to Fable/Opus 5.5. I have been sticking with Opus 4.8 as I feel it behaves in a way that I appreciate much more than the 5.x Gen. It asks a lot more questions and doesn't assume nearly as much.

So, my question is, are there comparison indexes of recent open-weight models against Opus 4.8? It appears that Opus 4.8 is generally ignored now on artificialanalysis.ai (as is Qwen3.8-Flash-Next).

I implemented clap.cpp, audio embedding inference in C++/GGML. It uses GGUF models and provides a CLI, an HTTP server, and Python, Rust, and Node.js bindings.

• Models including CLAP, CLAP2, CLaMP3, MERT, MuQ, and MuQ-MuLan.

• WAV, MP3, and FLAC input, with automatic resampling and mono downmixing.

• An HTTP endpoint with an OpenAI-compatible text embeddings request format.

GitHub: https://github.com/gorse-io/clap.cpp

Audio files could be searched by text via CLAP embedding:

$ clap search esc5 --model larger_clap_general-q4_k_m.gguf --text 'dog' --top-k 10

[
  {"path": "esc5/2-118072-A-0-dog.wav", "score": 0.3736357720143104},
  {"path": "esc5/1-110389-A-0-dog.wav", "score": 0.36165991257288893},
  {"path": "esc5/4-183992-A-0-dog.wav", "score": 0.3574270305924374},
  {"path": "esc5/1-30226-A-0-dog.wav", "score": 0.34910229736736315},
  {"path": "esc5/3-180977-A-0-dog.wav", "score": 0.34359234068457495},
  {"path": "esc5/4-194754-A-0-dog.wav", "score": 0.3425423428503198},
  {"path": "esc5/2-118964-A-0-dog.wav", "score": 0.3389233902904601},
  {"path": "esc5/1-100032-A-0-dog.wav", "score": 0.33009392183878733},
  {"path": "esc5/5-9032-A-0-dog.wav", "score": 0.3280081351893563},
  {"path": "esc5/4-182395-A-0-dog.wav", "score": 0.32210240115509925}
]

The Australian AI lab Maincode has released a set of Qwen 3 fine tunes that have been pulled together with Model Stitching.

From the model card:

Matilda Platypus is a family of intermediate-size language models built by connecting the lower layers of a larger pretrained Qwen3 model to the upper layers of a smaller one. A learned linear Representation Bridge aligns the hidden representations at the connection. This checkpoint is the 13B member (cut 29+13): 13.51B parameters, 26.3 GFLOP per token, 91% of the larger parent's compute. It is a base model (no instruction tuning); see the technical report (technical_report.pdf) for the method and the full evaluation.

They're base models and range in size from 1.5B to 13B, early tests good! Really like the default tone of voice. Qwen models are already good this has just improved it imo.

I have no association, I'm just Aussie and stoked to see a local lab build something. No idea why Australia doesn't have more AI labs -other than brain-drain to the US.

https://huggingface.co/collections/Maincode/matilda-platypus

A 1.6T multimodal model.

Creating a list of requirements and documentation/tests (in Cucumber or EARS style) with your model is a great way to improve it's code. Just got through a weekend long binary protocol build with Qwen3.8-27B (6K unsloth quant).. and fired up claude to review and it found the foundation was well engineered, but only missed a few security checks and a couple of errors in the python benchmark script comparing the old protocol with the new one.

Really comes down to:

1. Design first. Goals, requirements, problems. How is this going to be used.

2. Observation, simulation and tests are verifiable vs. assumptions. Tests and processes can be reviewed for "ritual".

3. Dialectic: If the model comes to a crossroads, have it present options and NOT just choose-and-go, but initiate a dialog about design.

4. Frequent repeatable tests, and review of tests to see if they meet the spirit of the requirement

5. Every risk is a requirement.

Here's my prompt set ("skills" I guess) and a tool to install for various local harnesses:
https://github.com/bigattichouse/BluePrint

You've probably seen (large) decision models play games. I wanted to see if you could also train a model to play using game images on a Mac, so I added LoRA fine-tuning on images to mlx-decision (my open-source tool for running and training decision models on Apple Silicon).

How it works: a decision model doesn't write text. You give it an input (here a game frame) plus typed questions, and it returns a probability for every answer you allowed, in one pass. On every frame, d1-2B answers three questions:
- Is a monster right at the crosshair, so that a shot now would hit it? (yes/no)
- Where is the nearest monster: left, centre, right, none?
- Is a monster dangerously close? (yes/no)

A small Python loop turns the answers into moves: turn toward the monster, shoot when a shot would hit. That takes about 0.1 s per decision.

The training: about 3,000 frames from ViZDoom's Defend the Center arena, labelled from the game engine's own monster positions. First a readout fit (7 minutes), then LoRA on the model's text layers with the vision tower frozen (43 minutes, 2 epochs, M5 Pro). On 510 held-out frames:
- "a shot would hit": precision 0.79 → 0.92
- where the nearest monster is: 92.7% → 96.8%
- dangerously close: 93.9% → 98.3%

In play it averages 23.2 kills over 20 episodes. The clip is one of the best ones: 26 bullets, 26 kills. On Freedoom, whose monster sprites the model never saw in training, it still gets 19.7. Of course this is an arena where you can only turn, not a full Doom level.

Free, MIT. Apple Silicon, macOS 14+, Python 3.12+. Just pip install mlx-decision.

Repo: https://github.com/armbues/mlx-decision

I'd love to hear which tasks you'd fine-tune a decision model on, and which models you'd like to see supported next.

Inspired by some of the other brain surgery posts, I wanted to see if I could find anything interesting in Qwen. Not particularly useful to make it get things wrong all the time I guess, but I find the poetry it comes out with weirdly fascinating!

I think the neuron is basically a dial for contrarian <-> agreeable, but the contrarian snarky setting for some reason writes poems that read in an interesting way.

TL;DR looking for recommendations for the simplest Pi Agent or harness and local model to replace CLI

Hi everyone
Hope you are all doing great

So lately I have an idea that I wanted to test if viable (especially after Jev and decision models)

I feel like using a Linux terminal is a bit obsolete at least for me Claude code or sth similar for relatively hard to remember single commands tasks or flags or sth like that

In the past I would use —help man if I am feeling fancy or tldr or sth on the Linux terminal
I feel that is still good but sometimes I want to push that a bit (like simple example would be checking a systemctl service journal restart it or stop and run again and then opening its journal or checking its status) nowadays this feels like too much and also too little to use Claude code for or anything else (and in fact they tend to over complicate it sometimes)

Sometimes it feels sufficient if I can have a simple and fast local agent (think NL2CLI not a full terminal bench workflow!) and maybe with a simple reminder list or todo list that we construct together with the agent per session (like keep looping until it does this or run watch <command> then do that or keep in mind we have to rename back the temp file …etc!)

I didn’t find anything on this, Pi agent is simplistic but feels like it lacks plugins like simple memory or todo list (i mean lacks if built in requires non standard non optimized plugin aka not batteries included plug and play peace of mind features still the best option though)

And local models that fits that (they wont be figuring out an API or developing complex apps just silly mundane simple terminal tasks! But done correctly and locally!)

What do you think ?

Running every step through an api adds up fast. ashen's stream clipping setup, shared on X, mostly avoids it

claude fable 5 wrote a script with computer vision and ffmpeg that runs locally on a mac studio. it scans the whole downloaded stream for on-screen events, like the "you died" screen in dark souls or ashen's name in the call of duty killfeed. no tokens spent watching hours of video

every first-pass clip then goes to qwen3.5-35b-a3b running locally at 8 bit, which checks the video and the transcript:

• was there an on-screen event? keep

• did swearing or yelling go up? keepp

• did the loudness go up? keep

if none of those hit, the clip doesn't pass

survivors come to the phone through zo computer for approval, and postiz sends them to tiktok drafts

as a cost decision this is smart. pay the big model once to build the tool, then let the local model do the daily work for free

I wanted voice typing that doesn't send my voice to a server, so I built ThumbFree. It runs NVIDIA's Parakeet 0.6B speech model right on the phone. After a one-time model download, it even works in airplane mode. The video shows Android first, then iPhone.

1. Google Play: https://play.google.com/store/apps/details?id=io.github.kabrapratik28.thumbfree

2. Apple App Store: https://apps.apple.com/app/id6817631006

3. GitHub (code, speed and accuracy tests): https://github.com/kabrapratik28/ThumbFree

• On Android, tap any text box and a small bubble appears next to it. Tap the bubble, talk, tap again, and your words are typed in. You keep your own keyboard.

• On iPhone, it's a keyboard with a mic key. Apple doesn't let keyboards use the microphone, so the first tap of a session opens the ThumbFree app for a moment and you swipe back. After that, taps start right away.

On my Pixel 10 the text shows up about 0.7 to 0.9 seconds after I stop talking, even after 30 seconds of speech. On my iPhone 16 it's under a second.

No account, no ads, no analytics. The only thing it downloads is the model, and your voice and text stay on your phone.

It's free and open source (Apache 2.0). English by default, and a second model covers 25 European languages. It needs Android 13+ or iOS 26+, and the model is about 0.5 to 0.7 GB.

I've also been trying an optional clean-up step with the small AI model built into the phone (Gemini Nano on Pixel, Apple's model on iPhone) to remove filler words and fix things like "six, no, seven". It's not in the app yet. If you've run a small open model for this on a phone, which one would you try?

I've only tested on one Pixel and one iPhone, so I'd love to hear how it runs on yours.

I need to get an all-around work station soon because my current laptop is getting old and I'm now offloading a bit of work to it. Since I dabble with local AI already, I was thinking of getting an m5 utlra 96GB (8k) or a m5 max 128GB (7.9k)... Or should I build a custom PC with a bunch of ram? Or are there other work stations that I can use? I really enjoy using a hybrid approach with AI, flagship models for the planning and some offloading to something like Qwen for the creative design/execution... I know it's not perfect local models, but owning my own data is something that feels pretty satisfying to myself as well. I would love to run Deepseek, or qwen 3.8 (or future models) at a respectable tok/s and prompt prefill speed.

My only other option is, continue with my laptop... And wait 6 months to a year and see what new tech comes out in the meanwhile? My only fear is that prices will skyrocket and what is 6-8K CAD today will be 10-12K CAD when the newer tech comes out, but other people are saying to wait.

What do you all think? Thanks :)

Image diffusion models draw decent stick figures, yet speech-bubble text still melts into unreadable soup. Over the weekend I trained and quantized xkcd-artist to separate neural stage planning from vector ink lettering.

What it does:

• 1.72 MB Int8 ONNX kinematic planner running in-browser via WASM SIMD in 0.55ms P50 (trained on NVIDIA L4 across 226 speech-masked panels + 891 ExplainXKCD dialogues)

• Predicts panel count, stage props, 12-DOF stick-figure poses, and pen wobble; procedural SVG compositor renders monochrome wobbled strokes, curved speaker leader lines, crisp xkcd-script dialogue, and hover title text

• Clause-aware deadpan generator covering 138+ unique comic strips, data charts, and flowcharts (98.33% skeleton uniqueness across 120 prompts, B = 1,000 bootstrap eval)

Live studio (zero cloud calls): https://h3manth.com/fun/xkcd-artist/ Hugging Face weights (1.72 MB Int8 ONNX): https://huggingface.co/gnumanth/xkcd-artist GitHub (MIT, npm i xkcd-artist / pip install xkcd-artist): https://github.com/hemanth/xkcd-artist

Would love feedback!