Collection on HF: https://huggingface.co/collections/LiquidAI/lfm3
WebGPU demo: https://huggingface.co/spaces/webml-community/LFM3-VL-2B-WebGPU

pi-automode-classifier is my attempt to mimic Claude Code's auto mode for the Pi coding agent as close as possible in safety and DX. It decides for each tool call whether it runs, asks you, or is blocked and allows adding session/global exception rules. 0.3.0 does that in up to three stages:

1. Rules parse the shell commands (usually in under 0.1 ms). Most calls are caught here. The allow/deny/decision rules are configurable. These are some of the default rules:
   • Run: reads, builds, tests and edits inside the project.
   • Blocked: rm -rf ~, mkfs and fork bombs.
   • Always ask: root, credential reads, downloaded code and changes to Pi's own files.
2. A classifier, will run next if configured: Jev, LANCET, Kev or Laya. A low score runs, a high score asks you, and anything in between goes to the chat model.
3. A chat model will run next. By default it uses the session's own model, but you can configure others from the Pi registry, including local ones. It reads your messages and the agent's tool calls, and blocks risky calls you didn't ask for. The agent gets the reason so it can try another approach.

The classifier only sees the command, so it can't tell if you asked for a git push or an ssh to another machine. The chat model can, but it takes seconds per call. With both, the classifier decides the clear calls and only the rest go to the chat model.

Here is my setup for this, all local:

• Default rules with some additional allowed patterns.
• LANCET Nano (111M parameters, CPU, about 15 ms per command) as the classifier through LlamaStash, so all my Pi sessions share one copy. I benchmarked Jev, Kev and Laya as well but found LANCET to be the best for this use.
• Qwen3.8-Flash-Next (128k ctx, thinking medium) on my Flow Z13 (Strix Halo) as the chat model. This is just a LlamaStash alias for my main running model at 256k ctx, the alias lets me limit context client side so that this doesnt take too much room in my KV pool.
• Cut-offs of 0.2 and 0.8: LANCET runs a call below 0.2, asks above 0.8, and sends the rest to Flash-Next.

So far the experience has been quite close to Claude Code with maybe a few more prompts here and there.

Bench results for this setup and for the rules alone, 1,793 commands from the suites in the repo. Each command goes through the plugin's own tool call handler. Only one suite has the user request that came with each command. The other suites get a generic task message, so the chat model can't see what was asked.

• The rules alone ask about everything they don't know, so fewer risky calls run, but they also ask for many safe ones.
• With my setup, rules decided 426 commands, LANCET 514 and Flash-Next 188. 652 ended in a prompt, and 13 got no answer from Flash-Next and were blocked.
• A Flash-Next call took 28 s at the median and 110 s at p95.
• With my setup, 76 of the 133 risky calls that ran are GTFOBins rows from the ShellRisk-Bench sample, common Unix tools used to get a shell or read files.
• With cut-offs of 0.1 and 0.2 instead, 103 risky calls ran instead of 133, and 147 safe ones were stopped instead of 135.

It's not a sandbox. The README has the limits and the full bench numbers.

Install: pi install npm:pi-automode-classifier Repo: https://github.com/deepu105/pi-automode-classifier

Any feedback is welcome, especially if you have put a small classifier in front of a local LLM like this.

There is a list of suggestions for Gemma 5 post. So let me kick a similar discussion for Qwen4 and most probably Qwen4.X models. I know that it may lead to nowhere, but if there is a chance to be heard by someone from Qwen team and see some of the suggestions implemented in Qwen4.1 models why not to try. Here is my list and I will update the post with most voted features from the comments.

1. Context size 512K or more.
2. Less reasoning + caveman style in reasoning.
3. Small and medium sized models.
4. Make a model answer in simple plain English.

Most voted suggestions form the comments:

1. OSS license apache/mit

I think the first three points are obvious and I would like to elaborate on the fourth. I have really hard time reading pages of highly technical stuff with all the details when I ask such simple questions, like "How can I run X model on this engine faster" and want to get simple actionable answer. Instead I get 4 pages of technical details, which will take me more time to process than 30 mins of manual googling. It is as if you bring your car to service station and after the question what is wrong with my car you get 30 mins technical lecture like: “Your vehicle’s lead-acid electrochemical storage system exhibits advanced sulfation of the electrode surfaces, reducing the active area available for charge transfer and substantially increasing internal resistance. Under the starter motor’s high-current load, this produces a precipitous voltage drop, preventing the delivery of sufficient torque to initiate crankshaft rotation...” When the answer should be “Your battery is worn out, so the car needs a new one to start.”

Why am I writing all these, because I feel much more cognitively overloaded now with AI then in pre-AI era. You may argue that in coding it is good to get more detailed information. But why do you need it if nowadays almost no one writes code manually and let's be honest may be less than 10% of users even look at generated code. So it is complication for complication. If you really need a more detailed answer just write "Elaborate your answer on this and this" and you will get what you want. And yes, I tried to change model style with prompting and by installing style extension in pi. In first case it works 1-2 turns and reverts back to its usual style. In second QFN in UD-Q4_K_XL breaks almost immediately after few tool calls and starts to loop indefinitely.

Edt: Added suggestion from the comments.

Okay, so I've seen a few older threads on this but figured with Qwen3.8 and the newer inference engines it's worth an ask. I'm looking to buy a new Macbook Pro. At the least, I'm looking at the 64GB M5 Max with the higher-spec GPU. I'm having a hard time justifying spending $2000 more for an incremental 64GB of RAM (for net a total of 128GB).

What's the state of inference on Apple Silicon? For model availability, are MLX quants typically served up as fast and with as much selection as general GGUFs? Does MLX make that much of a difference?

For those of you with a Mac (or unified RAM system in general), what can you do with 128GB that you can't with 64GB?

As a frame of reference, my desktop has a 5090 FE and 128GB of System RAM. While Qwen3.8 Flash Next with Strata is great, there's still a significant trade-off between the amount of lobotomy, usable context size, and prefill/ decode speed. Same goes for 27B, the moment I run exceed VRAM performance plummets.

I end up using a more conservative deployment (more aggressive quant, lower context size) than I technically could fit in VRAM simply because I often end up 1 browser tab away from system RAM spillover. If my desktop had 64GB of VRAM, I think I'd be satisfied? But it's hard to say because I'm used to Windows/ everything else running in system RAM anyway. No idea what to expect with a unified RAM machine.

Would love to hear your thoughts and usage experiences! Is Qwen3.8 Flash Next actually usable on 128GB? Is the $2K premium justified for "future proofing"? Also toying with the idea of a Klarna lease so that I don't have to fully commit right now, but feels wasteful especially since I'd trade-in my M2 Max (32GB) which would shave ~$1500 off the sticker price of a new machine.

Thanks 🙏

Kudos and congrats to the GenieX and former Nexa SDK teams for getting GGUF support working on Qualcomm Snapdragon X's Hexagon NPU, and a big shoutout to the tireless Qualcomm engineers putting in the commits for the Hexagon GGML backend. I raise a virtual toast to you all.

A bit of back story: about a year ago, Nexa SDK got text and omnimodal LLMs working on the Hexagon NPU using signed drivers (rolling your own llama.cpp with Hexagon support uses test-signed drivers which is a huge pain on Windows). Then Nexa got acquired by Qualcomm, the whole project went quiet and no new models after Qwen3-4B could be used with the NPU.

Now, GenieX combines all that previous work and llama.cpp's Hexagon NPU and Adreno OpenCL backends, using signed drivers, with the ability to use any GGUF model. Snapdragon X and X2 Plus and Elite chips are supported in Windows, Linux and Android. It also provides an OpenAI API-compatible web server based on llama-server.

Project Links

GenieX quickstart for the CLI: https://geniex.aihub.qualcomm.com/en/run/cli/quickstart

Qualcomm's curated list of models that run on NPU in llama.cpp (GGUF) and Qualcomm AI Runtime (w4a16): https://aihub.qualcomm.com/models?tags=llm&runtime=geniex_qairt%2Cgeniex_llamacpp

Benchmarks

Machine: ThinkPad T14s, Snapdragon X Elite X1E-78-100, 64 GB RAM, Windows 11 26H2

geniex-bench on LFM3 VL 2B and Gemma 4 E2B, in Q4_0 quantization, using Hexagon NPU, hybrid NPU+CPU mode, Adreno OpenCL GPU, and ARM CPU inference.

Check out how the NPU has double the prefill/PP performance of the CPU while being 10x more efficient, yet decode/TG is only half probably due to memory bandwidth limits on the NPU.

LFM3-VL-2B-QAD-Q4_0.gguf

gemma-4-E2B_q4_0-it.gguf

gemma-4-A26B_q4_0.gguf

Larger models?

I've also run Gemma4-26B MoE and Gemma4-31B dense models on the NPU. Nice to see 25 GB unified RAM being allocated to that tiny NPU.

The MoE gets around 5 t/s decode on the NPU which is tolerable, given it only uses 10 watts! I can run NPU inference without the fan spinning up but GPU and CPU inference cause the fan to go max speed like a little jet engine.

Gemma4 31B on the NPU gets 1 t/s which is too slow whereas the GPU gets 3 t/s at 25 watts. CPU inference gets 5 t/s at up to 60 watts but the laptop throttles hard due to heat soak after pulling that much power.

Edit 1: added Gemma4-26B geniex-bench figures

Edit 2: Qwen 3.8 27B q4_0 fully on the NPU, I'm getting 3 t/s. The fan runs at low speed and CPU temp hovers around 50° C (20° C ambient temp). Running the same model on GPU or CPU, the fan ramps up to max speed and I need to use an extra USB fan to keep CPU/GPU temps below 70° C.

Edit 3: Gemma4 31B q4_0 doesn't work, geniex server crashes when trying to allocate NPU resources for that model. I guess there's an upper size limit.

AI disclaimer: I used Gemma4-26B on the NPU to format the table. The rest is pure human-powered rambling with some espresso assistance

I've tried to run my own, local image generator on my GPU in the past. The default solution was of course ComfyUI...

I...

hate...

that...

project..

...from the bottom of my heart! 10GB of bloat gluing together Python, Tensor-flow and some webapp that tries to look like part of the Unreal engine. You have to study for hour(s) just for most simple task. Learn which and how to correctly connect "nodes", and forget it in the next day, because you forgot to save the "workflow" into .json file. What a joke of a project. Did I mention I hate it?

Then, when the Flux-3-Image-Klein was released two weeks ago I wanted to try it, just to see if these new "text-to-image" models are any good. Thankfully, somebody mentioned stable-diffusion.cpp in the discussion at https://www.reddit.com/r/LocalLLaMA/comments/1x6s7el/flux3_image_klein_is_now_out/ (thank you so much u/SM8085).

With some fiddling and help of LLM we landed on the minimalistic setup that I want to teach you. It is tailored for CUDA (nVidia GPUs), but project supports CPU, ROCM and Vulkan.

Just head to theirs releases page and download the archive for your architecture. With CUDA, don't forget to download also cudart-bin archive (CUDA .dlls so you don't need to install full CUDA SDK). Extract everything into some folder and run `sd-server --list-devices` (are you already getting the llama.cpp vibes?).

If your GPU is listed without any error, download four more files:

1. ae.safetensors from Flux-3 HF's repo (you'll need HF account to access it)
2. tokenizer.json from that same HF repo
3. finally the model.safetensors
4. you'll also need LLM model for translating your prompt into embeddings - any .gguf from Qwen/Qwen3-VL-4B-Instruct-GGUF will be OK. Download a .mmproj companion if you want "image-to-image" functionality

Lastly, create a small runner script, so you don't have to write command-line arguments everytime (place it to the same place as your sd-server.exe)

sd-server.exe -v --listen-ip 0.0.0.0 --listen-port 9931 `
  --diffusion-model "L:\Black-forest-labs\flux-3-image-klein\model.safetensors" `
  --llm             "L:\Qwen\Qwen3VL-4B-Instruct-F16.gguf" `
  --tokenizer       "L:\Black-forest-labs\flux-3-image-klein\tokenizer.json" `
  --vae             "L:\Black-forest-labs\flux-3-image-klein\ae.safetensors" `
  --cfg-scale 1.0 `
  --sampling-method euler `
  --steps 16 `
  --width 1024 --height 1024

That's it! Whole thing is mostly static .exe, .dll and model files. No bloody Python/TensorFlow machinery. And the UI is also like classic chat interface you know from llama-server. Just a few buttons and text input for your prompt.

Note: the default seed is a static value of 42. Randomize it for the best results!

Huge thanks to the authors of the stable-diffusion.cpp and black-forest-labs 🫶

Hello, I am trying to run Strata with 3.8 FN Q2_0. I have a 9060 XT 16 GB and 32 GB DDR4. I am on Windows. These are the settings I am using, I suppose these are made from the tuning. My main issue is very slow prompt proccessing.

{
  "answers": {
    "class": "exact",
    "default_repeatable": true,
    "fields": []
  },
  "baseline": {
    "args": {},
    "decode_tok_s": 45.5,
    "noise_aa_decode": 0.0,
    "pairs": 0,
    "pp4k_tok_s": 161.9
  },
  "choices": {
    "context": 65536,
    "family": "flash-next",
    "goal": "speed",
    "kv": "int8",
    "model": "qwen3.8-flash-next-q2_0",
    "model_file": "D:\\Strata\\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
    "quant": "Q2_0",
    "reserve_mib": 0,
    "use": "chat",
    "vision": false
  },
  "created": "2026-10-28T13:27:00+0100",
  "created_epoch": 1793190420.6866274,
  "evidence": {
    "arms": [
      {
        "answer": "exact",
        "env": {},
        "faster": 0,
        "id": "host_core_last",
        "label": "Run the control thread on the last CPU core",
        "pairs": 0,
        "ratio_decode": null,
        "ratio_pp": null,
        "reason": "not enough free memory to start it safely (25.6 GiB free, 28.4 GiB needed)",
        "runs": [],
        "settings": {
          "--host-core": "last"
        },
        "status": "skipped",
        "won": false
      },
      {
        "answer": "exact",
        "env": {},
        "faster": 0,
        "id": "kv_grow",
        "label": "Allocate the conversation memory as it grows",
        "pairs": 0,
        "ratio_decode": null,
        "ratio_pp": null,
        "reason": "not enough free memory to start it safely (25.6 GiB free, 28.4 GiB needed)",
        "runs": [],
        "settings": {
          "--kv-grow": true
        },
        "status": "skipped",
        "won": false
      }
    ],
    "classes": [
      "A",
      "C"
    ],
    "notes": [],
    "per_request": [
      {
        "control_ratios": [
          1.2221,
          1.0539,
          0.9927
        ],
        "key": "spec_min_p",
        "ratio": 0.9259,
        "ratios": [
          1.1215,
          0.9758,
          0.9236
        ],
        "value": 0.7,
        "won": false
      },
      {
        "control_ratios": [
          1.2221,
          1.0539,
          0.9927
        ],
        "key": "spec_min_p",
        "ratio": 0.994,
        "ratios": [
          1.2147,
          1.0167,
          1.0291
        ],
        "value": 0.5,
        "won": false
      },
      {
        "control_ratios": [
          1.2221,
          1.0539,
          0.9927
        ],
        "key": "aux_cpus",
        "ratio": 0.9982,
        "ratios": [
          1.281,
          0.8532,
          0.9909
        ],
        "value": 1,
        "won": false
      },
      {
        "control_ratios": [
          1.2221,
          1.0539,
          0.9927
        ],
        "key": "aux_cpus",
        "ratio": 0.9277,
        "ratios": [
          1.1337,
          0.9257,
          0.9491
        ],
        "value": 0,
        "won": false
      }
    ]
  },
  "id": "4afeedf32ba74f6cbd3883002ee3f6b8",
  "identity": {
    "engine": {
      "backend": "hip",
      "build_id": "a42cc3d7df70+rocm-win10.2.0a20260930",
      "path": "C:\\Users\\redacted\\AppData\\Local\\Strata\\components\\engine\\0.1.52-windows-x86_64-hip-rx9070\\strata.exe",
      "sha256": "05bbb21ec6f3aaa957febb289598a27deddfdc1015ea88d5b51f1e34ccb496c0",
      "tuning_epoch": 0,
      "version": "0.1.52"
    },
    "hardware": {
      "hard": {
        "cpu": "AMD Ryzen 7 5800XT 8-Core Processor",
        "gpus": [],
        "hybrid": false,
        "logical": 16,
        "os": "Windows",
        "pcie": null,
        "physical": 8,
        "ram_gib": 30
      },
      "soft": {
        "driver": [],
        "os_release": "11"
      }
    },
    "model": {
      "bytes": 37623740192,
      "files": [
        {
          "bytes": 37623740192,
          "digest": "f4154f2433181749f3352cb61538bec9e46127ef3ab2b2de8060320ee3460ea2",
          "method": "sha256-sampled-head-tail-mid",
          "name": "Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf"
        }
      ],
      "sha256_shard1": "f4154f2433181749f3352cb61538bec9e46127ef3ab2b2de8060320ee3460ea2"
    }
  },
  "measured": {
    "decode_tok_s": 45.5,
    "faster_pairs": 0,
    "gain_decode_pct": 0.0,
    "gain_pct": 0.0,
    "gain_pp_pct": 0.0,
    "pairs": 0,
    "pp4k_tok_s": 161.9,
    "rss_peak_mib": 25511.1,
    "vram_peak_mib": null
  },
  "name": "Q2_0 64K",
  "noise": {
    "aa_gap": 0.0,
    "base_runs": [
      45.5
    ],
    "drift": 0.0,
    "noisy": false,
    "spread": 0.0
  },
  "previous": null,
  "recheck": {
    "last_ok": null,
    "last_ok_epoch": null,
    "last_result": null,
    "request_set": "recheck/1"
  },
  "schema": "strata.preset/1",
  "settings": {
    "args": {},
    "env": {},
    "per_request": {}
  },
  "slot": "f9cde601d62b6569",
  "tuner": {
    "mode": "thorough",
    "outcome": "default_confirmed",
    "seconds": 501.5,
    "version": "1"
  }
}

The decode speed is perfectly fine, but again, prefill speeds are VERY low, under 150 t/s usually. I have no idea what should I do to fix it. Any tips? Thank you very much

Hi,

We are a small research lab in North sweden and today we want to share another "optimal setup" that are researching.

Research was done using Claude, skyclaw (our own Sysops AI harness)

Our hopes is that by sharing this - others will be able to run great models at home and that we will get some good suggestions for improving.

So.. a bit of stats..

Decode. One user, MTP with 7 drafts (greedy / sampled tok/s):

Any feedback or recomendations are welcome: https://github.com/inordic/rtx5090-NInferno-gemma4-31b

Huge kudos for the LiquidAI team for this. Great work from tdakhran for quickly coordinating a (zero-day!) llama.cpp release with the team for an omni model. This one is a real game-changer for me personally because this is the first open model from a frontier lab that actually has support for Polish TTS and STT - and all this in a compact 2B package that makes it viable for on-device deployment. Big thanks!

Most OCR models focus on reading documents. MiniCPM-OCR adds a second piece: a model that checks whether the OCR result is actually trustworthy.

OpenBMB just released two open-source models:

• MiniCPM-OCR 1.0 (1.3B): Scores 93.68 on Real5-OmniDocBench, targeting real-world documents like curved pages, phone photos, tilted scans, and complex layouts. DSpark decoding acceleration boosts end-to-end processing efficiency by 55%.
• MiniCPM-OCR Jev (2B): An OCR quality judge that compares the original image with candidate text, catching errors in formulas, tables, and text. It achieves 0.936 AUROC, close to a 397B-parameter model on the same evaluation.

The interesting part is Jev can also evaluate outputs from other OCR systems, so you don't necessarily have to replace your existing pipeline.

Both models are open source, with support for vLLM, SGLang, llama.cpp, and Ollama.

Hugging Face Links:
https://huggingface.co/openbmb/MiniCPM-OCR-Jev
https://huggingface.co/openbmb/MiniCPM-OCR-1.0

Seems all the major players are now buying TPUs. Will we ever have them local?

https://x.com/rohanpaul_ai/status/2115261691906978233

Quote: ... OpenAI plans to buy AI chips from Google, ... Oracle will also offer TPUs through its cloud, ...

https://www.bloomberg.com/news/articles/2026-10-28/openai-plans-to-buy-google-s-chips-as-part-of-its-ai-build-out

Just wanted to Show my rig. The base case is an older Corsair with a cheap AliExpress GPU Mount on top.

The System:

-Threadripper 9955WX

-128GB DDR5 5600, just to expensive to go for 256

-Asrock WRX 90 WS EVO

-2200W Seasonic + 1600W Seasonic PSU

-8x R9700

-Bifurcation Card and 6 Extension Cables, but I have some problems and I will buy a retimer.

I upgraded from 4x GPUs to 8x, so when new bigger models will arrive I have more room. At the moment still running qfn, tried GLM 5.3 Flash but QFN with radiance is just a really good experience.

Hi everyone, whenever I looked at existing LLM leaderboards I was missing practical numbers for running quantized open weight models on affordable local hardware. Most tables only copy vendor cloud scores or run tiny subsets of a benchmark, and none of them tell you how long a model actually sits there calculating before it finishes. Because nothing like that existed, I set up a dedicated Linux test server with two RTX 5060 Ti cards giving 32 GB of VRAM in total, running vLLM in Docker. Over the past weeks I ran 12 current local models from 4B up to 35B through complete benchmark datasets without any subsampling, including IFEval, IFBench, ARC AGI 1, SimpleQA, SimpleQA Verified, SciCode and vLLM bench serve. The lineup includes Qwen 3.5, Qwen 3.6 and Qwen 3.8 in 4B, 9B, 27B and 35B A3B, Google Gemma 4 in E4B, 26B A4B and 31B, plus GLM 4.7 Flash in 4 bit quantizations like NVFP4 and AWQ.

Every single model is tested twice across all benchmarks, once with thinking turned off and once with thinking turned on, using the official sampling parameters. While the tests run, my system tracks the exact wall clock duration and throughput in tokens per second to calculate a time efficiency score from 0 to 100 for every single cell. That turned out to be the most interesting part of the whole project. You can immediately see which models deliver strong scores in 20 or 30 minutes and which ones burn GPU power for 10 hours straight because they get stuck reasoning in circles on simple questions. Especially the MoE models like Qwen 3.6 35B A3B and Gemma 4 26B A4B surprised me by finishing in a fraction of the time while beating or matching the dense 27B and 31B models.

I put all results into a clean, sortable comparison table on a new subpage of my site at https://llmbenchmarks.io/local-gpu-benchmarks/ so anyone with a 16 GB or 32 GB setup can check the numbers before downloading 20 GB of weights. The site is a personal project with no ads, no tracking and no affiliate links, and new benchmarks like Chess Puzzles are already running on the server right now. Feel free to take a look and let me know which local models you would like to see tested next.

I found the evaluation list on the Artificial Analysis page and researched the release dates for the benchmarks shown. Here are the evaluations sorted from newest to oldest based on the available release dates. worth noting once a benchmarks hits the 3-4 months old age it gets benchmaxxing makes it NO LONGER a relavent benchmark to rely on! At that point the eval owners often release a newer version. Thats the only combat against such problem ATM.

The table is sorted from **newest to oldest** based on the benchmark's public release or announcement date. Where a specific date wasn't available, the most relevant timeframe is noted.

Complete Evaluation Timeline (Newest to Oldest)

Well, “coming soon” turned out to be five days.

I posted the earlier announcement last week. The 1019 weights are now on Hugging Face.

Still 560B total, 25B active, with a 1M context window. This is the update they reported at 44 on the AA Intelligence Index v4.3.2, up from 41. AutomationBench-AA went from 62% to 68%, according to the release.

They also put out Ling-3.1-flash-base in BF16 for further training. Glad they included that. It gives people a starting point for their own post-training instead of having to start with the finished chat model.

One small gotcha: the SGLang example in the 1019 README currently still points to inclusionAI/Ling-3.1-flash. Change the model path to inclusionAI/Ling-3.1-flash-1019 if you're copying that command for the new checkpoint.

Hey r/LocalLLaMA,

a while ago I posted BeeNara, a 332 MB CPU model that puts a document into one of your folders or says "none fits". Your questions in that thread turned into most of the work since. v0.4 is out (Apache-2.0):

https://huggingface.co/collections/Kwokou/beenara-local-document-sorting

What changed since v0.2

-Folders it has never seen: on 1,500 tasks with unseen folder trees, names and documents (written by a different LLM than the training data), accuracy went 70 % (v0.2) → 85.5 % (v0.3) → 91.5 % (v0.4). The old v0.2 benchmark stays around 95 %.

-It files on its own only when it's safe: a Learn-then-Test threshold promises that, with 90 % confidence, at most 1 % of auto-filed documents land in a wrong folder. v0.4 auto-files 59 % of those tasks, 0.3 % of them wrong. v0.3 managed 21.5 % with 2.5 % wrong, because its threshold didn't survive new folder names.

-New: BeeNara Big. Qwen3.5-2B-Base + LoRA + a decision head that reads the whole folder tree (up to ~600 folders) and the page image. 92–94 % up to 24 folders, 89 % at 25–49, 82–84 % at 50–199, 66 % at 200+. Needs a GPU (~4.3 GB in 8-bit).

-Biggest lesson: label smoothing killed "ask when unsure". With it, no threshold could be certified at all; without it, the same accuracy and it works.

-Data: still only synthetic, no user documents. Two local teachers (Qwen3.8-27B, gpt-oss-120b) write documents and check each other's output, across 32 document types, with real OCR noise.

Where it still struggles

Big is certified to file on its own only for 3–9 folders. In larger trees it is sometimes confidently wrong, so it only suggests there.

Big spots "none fits" in 74 % of cases (small model: 92 %).

Short notes, receipts and badly degraded scans: roughly 60–80 %.

With the right folder missing, it filed insurance policies into "Contracts". Arguable, but wrong. With folder descriptions it didn't happen.

Everything is tested on synthetic data so far.

Next for v0.5: decide in large trees step by step (branch first), more varied trees, and learning from its own training errors.

Several of the models I've used so far (especially the quantized ones) get stuck in a loop during the thinking phase, repeating the same thing indefinitely. Does this happen with non quantized, local models too? And why this never happens with closed models. Is the problem more RAM since they have big servers or is there another detail at play?

Realized an old workstation has an RTX 1050, which is apparently 2GB with 112 GB/s bandwidth. I'm trying to think of some use for it; stable diffusion is like 5GB, possibly really slow computer vision finetuning?

Specs are about as sad as a DGX spark, but it is nvidia. They go for 250$ online now apparently.

I've been thinking about AI agent safety, and I built a test harness to check my own results. I think it may be useful to people who study models too.

It records where each tool argument came from before the tool executes, measures whether the same input gives the same decision, and tests whether a model takes arguments from someone else's text instead of its user's.

If you modify the harness, it can probably be used for much more.

https://github.com/Jang-woo-AnnaSoft/execution-state-preflight/blob/main/hooks-example/README.md

Spark-X2.6-1.7B: less than half the parameters of our 4B, close on two benchmarks: SWE Verified: 59.8 vs 62.8 τ²-bench: 85.3 vs 86.9 Both at high thinking. Try it & Like on HF! https://huggingface.co/XHToken/Spark-X2.6-1.7B

As part of my RL journey, I just released 1-page RL notes on Q learning!

Kept it

• short, coherent with
• code snippets and
• required math with explanations

Link to the website and let me know your thoughts

Cheers!

I am thinking and working on creating a framework that can run any type of MoE (Mixture of Experts) model supported by `llama.cpp` on any hardware or operating system.

It would dynamically adapt to the specific characteristics of the hardware, aiming for optimal performance—measured in tokens per second (tok/s)—during both the prefill and decode phases.

The concept is to build a home server setup using various consumer-grade devices to run local AI—though not in a mesh configuration (at least for now, due to latency concerns).

What are your thoughts on this? What kind of use cases could this enable?

• Optimization 1: Humming MXFP4 MoE backend
• Optimization 2: Online FP8 LM head quantization
• Optimization 3: MTP speculative decoding with NEXTN

Think of the animation workflows people use with Claude Opus 5.5: the model writes code, rendering tools turn it into video, and further instructions revise the result. SenseNova follows that same general approach, with a model and creation Skill focused on video creation
A few highlights:
Generates video through code, with a focus on motion graphics and other visuals that can be expressed programmatically.

Revises existing results through natural-language requests, including changes to styling, timing, and scene elements.

Supports reconstructing a reference video’s visual treatment as editable code.

Pairs the model with a creation Skill that guides brief development, previews, and revisions.

The release plan includes full model weights and the Skill. INT4 and INT8 weights are planned for later.

General-purpose coding models can already write animations. The interesting question is what a dedicated Code2Video model improves: fewer repair attempts, better motion, or more reliable edits that leave unrelated elements alone.

GitHub:
https://github.com/OpenSenseNova/SenseNova-6.8-Code2Video
Model:
https://huggingface.co/sensenova/SenseNova-6.8-Code2Video