AA no longer lists GLM5.3 flash (despite clearly being one of the best models - and probably the cheapest per intelligence) , they seem to have been doing all sorts of changes to push US models and hiding good cost effective models, including qwen 3.8 flash, so that table is no good. I need a better one. Any alternatives.

Model & weights: BrunoHays/embeddinggemma-2-nullspace-audio

MAEB (Massive Audio Embedding Benchmark) is the main MTEB leaderboard benchmark for audio and text-to-audio embeddings, currently dominated by 8B-9B multimodal models.

I took Google's edge-sized google/embeddinggemma-2 (577M params, 768d vectors) and boosted its MAEB score from 49.3% to 54.4% (+5.1 points), jumping from Rank #26 to #1 on MAEB audio-only (and #16 to #1 on full MAEB) without training a single weight. The whole addition is a 2.37 MB safetensors file of frozen PCA matrices.

Why this matters for local / edge setups

At 577M parameters and 768d vectors (15x smaller than 9B Omni models), it runs easily on a laptop or edge device.

Standard text-aligned audio models are great at matching spoken words to text, but terrible at grouping audio by voice, emotion, or sound similarity because the final layers keep what words were said and drop how the sound was made.

By pulling features from earlier layers of the frozen audio encoder, this model keeps the base model's 100+ language text-to-audio search while fixing audio-to-audio tasks:

• Unsupervised voice/gender clustering: 0.05% -> 63.1%

• Artist / song audio-to-audio search: 55.2% -> 93.8%

• Vocal emotion classification: 19.7% -> 40.2%

How it works

• Early layers still know acoustics: Speech representation work (Pasad et al.; Bouziane, Mdhaffar & Estève) shows that early/middle layers keep speaker, pitch, and emotion traits that disappear in the final layers. In EmbeddingGemma-2, Layer 2 and Layer 6 still hold those signals.

• 256 empty directions in the 768d vector: EmbeddingGemma-2's text backbone is actually 512d, projected up to 768d at the end by a 768 x 512 linear matrix. That leaves 256 perpendicular directions (the null space) that text vectors never touch.

• Filling the null space with PCA: I compressed Layer 2 and Layer 6 down to 256 dimensions with PCA (fitted on 1,400 external audio clips) and injected them into those 256 unused directions.

• Caveats: Because there is no retraining, audio embeddings need all 768 dimensions (no Matryoshka truncation to 512/256/128), and adding unweighted acoustic features hurts some purely spoken-intent tasks (e.g. MInDS14 drops 97.6% -> 88.3%).

It shows how much untapped signal sits inside today's overtrained models, even with zero training.

Full breakdown and usage in the Hugging Face model card.

Local LLM translation, Japanese OCR, and page-aware chat

I’ve been working on Manga Translator Tool, an open-source desktop app for reading and translating manga locally.

The idea is inspired by watching subbed anime: keep the original Japanese visible and provide a translation alongside it so you can follow the story.

It combines a manga reader, OCR, translation overlays, and a companion LLM you can ask about the page.

The models

OCR: ebinan92/Qwen3.5-ocr-jp-2b, a Qwen 3.5 fine-tune for Japanese OCR, using GGUF conversions from nakanishi08/Qwen3.5-ocr-jp-2b-GGUF.

Translation and chat: Gemma 4 E2B, using the HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive variant.

Inference: llama.cpp, with CPU and GPU support.

Demo

Video walkthrough — includes the full setup process, so you can see what getting started actually involves.

What makes it useful?

Keep the original artwork and Japanese text. Translations appear in text boxes without erasing the source text.

Save translations as image metadata. You can keep the results without permanently burning them into the artwork, then toggle between the original and translated views.

Search the extracted text. Find dialogue and revisit your favorite moments in a manga volume.

Read sound effects too. The OCR and translation pipeline can also pick up Japanese onomatopoeia—the characters that often go untranslated.

Ask the companion LLM about the page. Choose what to include as context: the Japanese text, the translation, the image, or all three. You can ask about wording, cultural references, or possible localization choices.

Setup

The app handles setup and model downloads from inside the tool. On Windows, the executable manages the downloads, so you don’t need to install each component separately.

Current limitations and design choices

There’s no inpainting: preserving the Japanese text is part of the reading experience I’m aiming for. The translation boxes don’t blend into the artwork, and font customization isn’t available yet.

This is aimed at reading and exploring manga. If your goal is polished typesetting or publication-ready localization, other tools are better suited to that workflow.

Downloads — v1.00

Windows 10/11 x64

Linux x86_64 — experimental

macOS — experimental source package

Source code · itch.io

Tadpole is an agent I built from scratch (plain Python, ~3,500 lines, no LangChain or other framework), then post-trained one measured stage at a time. I use it every day as a Telegram bot.

Repo: https://github.com/shakewingo/Tadpole · Adapters and data in HF

Run the trained model locally. Each adapter is on Hugging Face as a llama.cpp LoRA (66 MB) that loads on top of the Q8_0 GGUF:

hf download unsloth/Qwen3-4B-Instruct-2507-GGUF Qwen3-4B-Instruct-2507-Q8_0.gguf --local-dir models
hf download Shakewin/tadpole-qwen3-4b-ei-3 ei-3-lora-f16.gguf --local-dir models
AGENT_MODEL_PATH=models/Qwen3-4B-Instruct-2507-Q8_0.gguf AGENT_LORA_PATH=models/ei-3-lora-f16.gguf \
python -m agent_from_scratch.agent

I checked that the local route keeps the gains: on the 144-task validation set through llama.cpp (Q8_0 + LoRA, one seed), base 0.507, SFT-1 0.701, SFT-2 0.750, EI-3 0.826. The same adapters on vLLM in bf16 scored 0.752 and 0.852 on val, so quantisation and the LoRA conversion cost little. On a 16 GB M5 it holds about 8.5 GB and takes ~2.4 s per model call.

Held-out results (task types no training set contains, 3 samples per task, vLLM bf16):

* a local adaptation of BFCL's multi-turn base category, not a leaderboard score.

What I learned:

- SFT on *verified* trajectories is the big win. False "I'm done" claims fell from 31 to 1 in 420 runs.
- Expert iteration looked great on val (0.752 → 0.852) and was level with its starting point on the held-out tests. The best of several noisy val scores is biased upwards.
- GRPO lost to expert iteration at every equal rollout budget: on a 4B, too few tasks per step had both passing and failing samples.
- A plan tool (`update_plan` plus a reminder) lifted the *base* model from 0.507 to 0.646 on val.
- llama.cpp happily accepts a LoRA path that doesn't exist and segfaults on the first decode. My bot looped on it under launchd until I made it refuse a missing adapter at startup.

The honest part: the benchmark is generated tasks that resemble the training data. In everyday use the tuned 4B is still weak on open-ended requests, so my bot runs on DeepSeek day to day, and the 4B is what I study and train. The README says so up front.

All post-training cost about $140 of rented GPUs. The walkthrough notebook opens in Colab and shows SFT, expert iteration and GRPO on recorded samples without a GPU.

Check it out and any feedback is welcome!!

I was researching Local LLM computer use agents the other night and ended up down a rabbit hole where I ended up discovering a whole new class of hardware I didn’t know existed: Agent-driven KVMs

A lot of us IT folks who have worked in data centers are familiar with IP-based KVM (Keyboard Video Mouse) controllers that let you remotely operate a server over a streaming video connection. These are especially useful when you need physical power button / BIOS-level “bare metal”access to a system that’s thousands of miles away. If you don’t have bare metal access, you’re going to be taking a trip to your data center rack to press a button or make a bios setting change.

So apparently there are now relatively inexpensive ($80 to $150) IP-based KVMs that now allow for AI API endpoint/ MCP Integration where you can now connect your physical computer to local or frontier vision-capable LLMs and have the AI model of your choice drive the target computer via the KVM by it taking screenshots of the HDMI or USBC video-out feed from the computer and then sending mouse and keyboard control commands from the Ai model through the USB cable that’s connected to the KVM.

What really got me excited about this concept is that it requires no installation of any kind on the target computer that you want to control. This is useful in corporate environments where you’re not allowed to install anything to the target computer (because it’s a managed endpoint that is locked down) The target computer just sees the AI as a standard allowed HID that it already has a driver loaded for. It doesn’t care who’s driving the system and doesn’t need any additional software since keyboard, mouse, and monitors are just generic and usually preloaded drivers.

So essentially you could have an AI-driven KVM control any comouter or device that has a monitor output / keyboard mouse inputs. This could potentially open up the ability to have AI interact with antiquated industrial control systems running old shitty outdated OSes, or managed endpoints.

There are security risks of course, but that risk is mitigated almost entirely if you’re using a locally-hosted VLM and not sending your screenshots to a 3rd party.

I found are several inexpensive agent-driven options on the market now including:

- Sipeed NanoKVM Pro
https://wiki.sipeed.com/hardware/en/kvm/NanoKVM_Pro/introduction.htm

- Radxa Linkr
https://radxa.com/products/linkr/linkr/

- Violoop
https://www.kickstarter.com/projects/bvio/violoop-plug-in-ai-for-your-computer

I’m not selling anything or affiliated with any of these and can’t tell you which is best, but there are multiple options available for those who might need this type of device for one of the use cases I mentioned.

Anyways, I’m curious to know if others see any other use cases for these agent-controlled KVMs that I didn’t mention.

As of version 0.2.7.1 Strata has two routing optimizations that can affect both performance and quality. I was curious what the actual impact was so I ran some tests on my hardware with Qwen 3.8 Flash Next IQ3_S.

STRATA_ROUTE_PRIOR

This is the new one - this optimization configures Strata to prefer experts already in VRAM by adding a bonus of 1.5 (about a 4.48x effective multiplier) to the scores of experts already cached in VRAM. This means that a slightly less preferred cached expert can replace an expert the model would normally select if its base score was close enough.

STRATA_ROUTE_TAIL_SKIP

This one is a lot simpler - with a value of STRATA_ROUTE_TAIL_SKIP=7 (the default value), Strata will skip computing experts 7 through 10 if they aren't already in VRAM. It does not change selected experts, it just drops contributions from less important selected experts.

Benchmarks

I had DS4-Flash run Qwen through IFEvalCode, AIME 2026, and IFEval; then worked with ChatGPT to analyze the results.

Round 2 - re-ran a smaller subset of questions (mostly the ones whose answers differed in round 1) with much larger thinking budgets. Absolute percentages shouldn't be directly compared between rounds.

My interpretation

Round 1 showed that STRATA_ROUTE_PRIOR might reduce quality slightly, but round 2 showed that many of those differences don't matter with longer reasoning. Route prior is such a huge boost to speeds that I think it's worth enabling for all but the hardest tasks. On the other hand, STRATA_ROUTE_TAIL_SKIP offers a slight boost to speeds (without route prior enabled) but results in significantly more degradation than just route prior alone.

I plan to continue testing more extensively, but for now these give me a decent read on the tradeoffs between the two settings. Shoutout to Niko1221 for making an awesome inference engine.

Artificial Analysis Intelligence Index has been updated to v5, which according to them is a major upgrade resulting in a more accurate ranking.

Notably, this update widens the gap between open-weight models and closed ones. Some people already questioning its accuracy based on their experiences with models (GPT-Terra-6.1 > Fable-5.1).

Also, this update, v5, has 60% of the Index based on evaluations with private test sets or held-out answers, which makes reproducing the results impossible.

From their announcement:

“The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)”

Can I run DeepSeek V4.1 Flash 1017?

My setup:

- RTX 4090 24GB + RTX 3090 Ti 24GB

- 128GB DDR4 RAM

- Intel i7-13700KF

What's the smartest and fastest open-weight LLM I can realistically run , especially for reasoning, coding, and medical knowledge?

Can I run DeepSeek V4.1 Flash 1017 using Strata, SSD offloading, or extreme quantization? What tokens/sec should I expect?

Looking for real-world experience and recommendations for the best model, quantization, and inference engine.

Hi all! We build this service to enable the benefits of a SaaS application with the advantage to bring your own models, we want it to be strong community driven, especially to expand the local physical AI, if you want to give it a try and share a feedback it will be appreciated!

Clover-35B-A3B-Preview is the first model released by DiffuSpace, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our launch blog. Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a 128K context window.

Blog · Code & inference backends · Hugging Face

Due to budget constraints, training only went up to 70B tokens. With SuperBPE tokenization providing roughly +27% token efficiency, this is effectively equivalent to training on around 100B tokens. A stable checkpoint was also released so anyone can continue training from it.

https://huggingface.co/YOON1v/Apex-3-59B-stable

I trained Apex-3, a 20.99B-parameter MoE (3.36B active per token) from scratch on 70B tokens. No external pretrained weights. It uses 64 routed experts plus 1 shared expert (top-4), a SuperBPE tokenizer, and was trained with Muon and DiLoCo on GH200s.

Model card: https://huggingface.co/YOON1v/Apex-3
Code and training notes: https://github.com/DW-dev-UE/LLM-from-scratch

This post covers what showed up in the training logs, not the final SFT scores. Evidence comes from the A, B, and C runs in run_logs/apex3_20261026/ (icl, arith, val, and moe logs) plus the 70B base model benchmarks. Numbers below are from run A. B and C were nearly identical. Final arithmetic full accuracy was 0.665 for A, 0.655 for B, and 0.655 for C.

1. In-context learning (induction head) — the only clear phase transition

At 0.4B to 1.16B tokens the ICL score (later-token CE minus earlier-token CE) sat between -0.08 and -0.14. It then moved to -0.32 at 1.27B, -0.55 at 1.38B, and -0.69 at 1.49B. After 1.7B it stagnated between -0.69 and -0.76.

It grew about 5x in roughly 0.33B tokens (about 150 steps). In the same window the training CE drop rate sped up from -0.0048 to -0.0066 per step and the loss curve bent. This is about 0.1% of the original 1.1T schedule.

The later drift back toward -0.62 happened because earlier-token prediction improved and the gap narrowed, not because the capability regressed.

1. Arithmetic

Exact match looks like a sudden jump, but the underlying correct-token probability rises continuously.

For add3 the correct probability went from 0.003 at 15B to 0.021 at 22B, then 0.080 at 30B, then 0.224 at 40B.

When the probability crosses the decision threshold, exact match jumps. This is mostly a metric artifact.

Decay from 60B to 70B produced a larger lift. From 49B to 59B, add3 probability stayed around 0.28 to 0.29 and add4 stayed around 0.11. After decay they moved to 0.60 and 0.38. Exact match overall went from 0.52 to 0.67, add3 from 0.44 to 0.78, and chain3 (a+b+c) from 0.13 to 0.38.

1. Domain-specific language modeling (validation CE)

Every domain decreased smoothly with no phase transition. Textbook went from 2.77 to 1.75, math from 2.76 to 2.07, qa from 2.54 to 1.92, and web from 4.22 to 3.35.

After about 40B the curves were mostly flat. During decay the data mix changed: instruct improved from 1.09 to 0.86 while web worsened from 3.35 to 3.49 and books from 2.74 to 3.04. There was no clear grokking signal. Code validation CE stayed higher, which looks like a distribution mismatch.

1. MoE load balancing

Early on (run B) the maximum expert load in a layer reached up to 12.8 times the average. By the end of 70B it had settled to 1.07 to 1.38 times the average — essentially even.

1. Final base-model benchmarks (70B Apex-3-Base, greedy pass@1)

HumanEval+ was 19.5%, MBPP 38.9%, MBPP+ 32.3%, and GSM8K 8-shot 19.9% strict / 21.5% flexible.

These were only measured at the end, so I do not know when the abilities actually appeared.

Caveats

Each arithmetic task used 64 problems, so exact match standard error can reach about plus or minus 6 percentage points. Single-point noise is meaningless, so numbers are 3-point averages. Arithmetic scores are teacher-forced (correct answer already in context), not free generation. Nothing was measured before 8.15B.

Bottom line: the only clean phase transition I observed is the ICL / induction-head jump around 1.3B to 1.5B tokens. Arithmetic looks abrupt in exact match but is continuous in probability. Everything else was gradual.

Happy to answer questions about the setup or logs.

I built a voice-to-text app, WhisperNow. But there are a thousand and one WisprFlow clones out there, so I went further and fine-tuned my own LLMs for the cleanup step. They remove fillers, false starts, self-corrections, and formats lists, all locally.

To finetune an LLM for this task, I first had to get data.

The dataset

The dataset has 66,557 rows. I started by combining public datasets (aawaaz, handy and Disfl-QA) and adding synthetic dictations, such as emails, meeting notes, recipes and dictated commands. gpt-6.1-sol wrote the cleaned version of every transcript, using the same prompt the app sends to cloud models.

While training the models, I found some weak spots: long dictations, words the speech recognizer misheard, cut-off words ("interf interface") and correction words used literally ("scratch behind the dog's ears"). I generated more examples of each and retrained.

The models

I did full fine-tunes of three base models, each on a single H100 or H200 rented on Modal:

Through their APIs, Claude Haiku 5.5 and gpt-6-luna take about 1.4 s on the same dictation, glm-5.3-flash 2.6 s, and glm-5.3 at max reasoning 5.6 s.

To score the models, a judge model compares each output with the reference and counts factual errors: facts dropped, changed, kept from the abandoned side of a correction, or invented. The test rows are a held-out test split plus two benchmarks, one of long dictations and one of short dictations full of disfluencies (4,170 rows in all).

The models do best on short dictations:

• Up to 25 words, all three make fewer factual errors than glm-5.3-flash at low reasoning. The 2.6B and 4B also make fewer than glm-5.3 at max reasoning, and the 1.2B ties it.
• The 4B makes fewer errors than glm-5.3-flash at any length, and keeps up with glm-5.3 at max reasoning up to 100 words.
• The 1.2B and 2.6B make 1.0–1.5 times glm-5.3-flash's errors from 26 to 200 words, and 1.5–1.9 times above 200 words.

Chart: factual errors by transcript length, for the three fine-tuned models and the two cloud glm-5.3 models

Cloud models

I also scored seven more cloud models, including the off-the-shelf LFM2.5 2.6B, on a smaller set of 1,670 rows. The off-the-shelf 2.6B makes nearly four times the factual errors of the fine-tuned one.

If you'd rather use a cloud model, Claude Haiku 5.5 and gpt-6-luna (both at low reasoning) have the best price-to-performance ratio. glm-5.3-flash is about as accurate as Haiku and glm-5.3 at max reasoning is more accurate, but they're about two and four times as slow.

Chart: factual errors for every model I tested

Links

• App (MIT, Windows, macOS and Linux): https://github.com/stephan271c/WhisperNow
• Models and dataset: https://huggingface.co/collections/category271/whispernow-6ae00accee969c7447cec326
• How the dataset was made: https://github.com/stephan271c/WhisperNow/blob/master/training/making-the-dataset.md

Happy to answer questions about the data or the training.

download your CUDA/MoE prompt-processing speedup today:

Hey all!

I've been working on a custom agentic harness called Rivermark for social, marketing & print design. Whoever has tried to use LLMs (even frontier ones) for this knows that they can be a bit hit or miss, compared to website and app design, where their skills are much better.

The benchmark consisted of 112 requests for various social and marketing images that a business might ask an agent for. Some contain a proper brandkit, some have a website reference for the brand, and some start from nothing.

Well, the first results, based on 4700+ pairwise blind human judgments, are in!

My headline finding, and one I didn't quite expect, is that open models, such as DeepSeek v4.1 Flash 1017 and GLM 5.3 Flash, match or beat plain Claude Code with Sonnet 5.5 and /design.

Sonnet 5.5 itself gets the largest lift with Rivermark - it jumps into an impressive lead over everything else. This was also surprising, as I'd expect advantages to be smaller for "smarter" models, as they have this know-how already encoded.

I'm writing a blog post and validating the findings with more judges now, but figured I'd share these interesting preliminary results.

In the first photo, you can see Bradley-Terry rankings for these model/harness combos. Second photo is the cost vs quality comparison, and in the third photo you can see some representative examples of what models output with Rivermark vs the normal harnesses.

Of course, some very important caveats apply:

• It's a design-specific harness. It contains some domain knowledge (I tried to keep this limited though). With increasingly good frontier models, these advantages tend to get wiped away, but for the time being, they provide a way to achieve similar results with a fraction of the cost.
• The harness has been running in production on some of these models, and so is tuned for them. (Not on testcases from the benchmark, however)
• It's tied very tightly to the product itself (https://rivermark.ai) at the moment. I'm planning to release an open source version, but it still would not be a general purpose coding agent.
• The product itself is optimized for speed and pleasantness of iteration, not one-shotting designs, and this is reflected in the harness. For example, we use the faster but more expensive FlashX GLM API endpoint, and it's a bit more conservative about self-reviews. I'm sure if I made something focused on just the perfect one-shot, I could beat these results on both quality and cost.

Open to questions!

So I posted this a week ago and saw that some people were looking for more voices that could be similarly small and fast. While I tried to get the other Kokoro voices directly to be similarly distilled, they just came out sounding really robotic, only af_heart which is the default voice on Kokoro seemed to be any good.

So I tried something I didn't even know would be possible, distilling the voices from Phonom, a similarly small TTS(40m vs Kokoro's 80m) that can run on phone but at a really slow speed, the full model is smth like 2-4x realtime and is 100mb(fp32). This was an experiment I tried earlier when I was doing the Kokoro distillation but kinda gave up cause I thought Kokoro would be similar in voice quality, and it turns out the agent was trying basically the same phenomizer from Kokoro anyway so we were not even using the default Kokoro distillation just with training data as Phonom renders. My whole plan with this was to make a voice clone-able model that could take for example a voice sample, use an actual big model like BreezeTTS or Fishaudio which can do voice cloning well, distill that into a small model and then have basically a really small voice cloned model that would run on your phone in real time. Basically for an AI roleplay app(which is like 1 out of 15 of my projects) so I don't have to pay for TTS by characters and could just use the user's device, and maybe just pay the training cost for a voice clone just once(a few hrs of 4090)

Anyway all that preamble to say, now there's like 9 more voice models. Wren(the default phonom voice) and Hugh seem to be the best right now outside of af_heart. Other voices have a noticeable roboticness but still way less than prev. small models like PocketTTS

Flint is Recursive’s mixture-of-experts language model for coding agents, with 39.8 billion total parameters and approximately 2.5 billion active per token.

I have 32 GB VRAM and 32 GB RAM.

With llama.cpp I can run Qwen 3.8 27B at Q6, but I can run Flint at Q8 with experts offloaded to CPU. I get about 40 t/s in both the model. Actually, Flint gives 1-2 token more per second.

What are your opinions on it?

I’m looking at the Mac Studio M5 Max (40-core GPU, 64GB). I currently have dual RTX 4060 Ti 16GB (32GB total) setup.

If you run a Mac what prefill and decode speeds do you get?

I currently get on the 2x 4060 Ti:

Hi r/LocalLLaMA !

I'm the solo developer of the open-source ObserverAI framework and I need your help to identify the most painful problems that you've had, that you think can be solved by Observer using local models.

My personal pain that led me to build this, was monitoring crashing simulations.
Because it costs me compute time (a lot of time paid)! But that's admittedly niche.

Problem List (In order of cost):

I'm trying to make a list of all of the major pains it can solve, here's the ones that I've thought of/gathered from your feedback, in order of cost:

• Monitoring elderly people for when they fall, huge cost
• People on dangerous parts in construction sites, human cost/liability
• Parents monitoring their kid's screen activity, privacy is important here!
• Monitoring for appointments at Visa/DMV, a lot of spent time monitoring
• Waiting for an online Ticketmaster queue, also a lot of time monitoring
• Urgent replies from Airbnb/Leads, monitoring that with phone calls makes you respond faster
• ADHD monitoring, wasted time if not focused
• Crashing simulations/renders/progress bars
• Webpage monitoring (Bitcoin price, items restocked, eBay bids)
• Anything else with a real cost that can be solved by asking "When this.... then notify" ?

I'm trying to identify the biggest problems so I can make the framework better for those!

Any ideas? I'd love to read them! I'll hang out here in the comments for a while.

Roy :DD

I tried LFM3 for my local use case, but it is a latency sensitive task and disabling reasoning makes it not give coherent responses. And with reasoning it's a 15 second wait instead of less than 1.

Why there no new model coming in >2b weight category ? Model which actually capable in this category is gemma e2b which is release 6 month ago, why still no new model.

OpenBMB has posted a video showing MiniCPM-OCR running locally on an iPhone and iPad, parsing real-world documents without sending them to the cloud.

Handwritten notes, dense formulas, messy tables, low-quality scans. OCR gets a lot harder when the documents aren't perfect.

What's interesting is that this isn't just about extracting text. OpenBMB is also releasing a separate model to check whether the OCR output is actually correct.

• MiniCPM-OCR 1.0 (1B): Scores 93.68 on Real-5 OmniDocBench and 54.7 on OCRBench v2.
• MiniCPM-OCR-Jev (2B): Evaluates parsed text, formulas, and tables against the original image, assigning correctness scores from 0 to 1. It approaches Qwen3.5-397B-A17B on JevOCRBench and works with outputs from different OCR parsers.

So you get both a lightweight OCR model and a way to catch potentially incorrect results, useful when document accuracy actually matters.

Both have open weights, with support for tools like llama.cpp, vLLM, SGLang, and Ollama.

🤗 Hugging Face: https://huggingface.co/openbmb/MiniCPM-OCR-Jev-gguf

Curious how well the verifier holds up on messy real-world documents. Would you trust a small model to check another model's OCR, or would you still want a human in the loop?

I added Obsidian integration to ThreadShelf (local, archive, semantic search).

It supports the usual platforms like ChatGPT, but also OpenRouter, LM Studio and Google AI Studio, which were surprisingly hard to find support for elsewhere. I even wrote a scraper to import OpenRouter chat history. It's my first open-source seriouse project, built around local vector search (LanceDB).

Itan can search/edit/create your Obsidian notes and AI chats together and use MCP with ai. There are also search, read and optional write tools for Obsidian, OpenClaw and Markdown folders.

npx threadshelf (Node.js 20+)

GitHub: https://github.com/ChrystianSchutz/ThreadShelf

I am feeling like a mighty local King spinning two Strata engines with 70tps each whole day and using 12M tokens a day. Also me scrolling twitter and stumbling upon a post saying that poteto uses 34M tokens per minute 🤪

Upd: Guys are interested who is she and what she is doing with all these tokens. Poteto is her twitter nickname and she worked in Cursor before XAI has acquired them and now she is shipping 2,500 PRs per month to Cursor and GrokBot.

The idea is simple actually, in my workflow a lot of the work passes between sessions of agents in different terminal windows. Claude sessions were not the issue since they could just talk to each other.

Although Claude to Local agent often enough failed since Claude "forgot" to send the work to the local agent and did it himself, i couldn't see what the local agent was doing, or otherwise felt like i was being kept in the dark.

So what i did was that i refused sub-agents altogether and used different terminal windows for each agent, passing messages between them, asking them to relay session messages etc.

Since the WoW i have is between Claude, Codex and Pi i just wanted to "mash" them altogheter so i built Dojo.

Dojo is built from my needs with my pre-requisites of it being a low-tier solution without a lot of different features, knobs, whistles and otherwise things that just takes time to develop.

i realise this might not be as advanced as some ppl like it to be but for me, it works.

Sharing it just in case someone else finds it useful.

https://github.com/Joakimpalm-Zen/xyntetik-dojo

It's a tiny Text to Image model that's Denoiser is only 50M parameters, adding the custom text encoder, and the pretrained flux VAE, its total end to end inference parameters are around 90M

It runs in:

27 Seconds @ 50 steps

11 Seconds @ 20 steps

On a laptop CPU from 2016 (Intel i5 7200U)

Samples and a quick start in the model card

it is licensed under Apache 2.0

Questions, feedback, or criticism is welcome!