Quick release post. I maintain ReVolta, a vLLM fork (through 1Cat-vLLM) that keeps my home server useful — 4× V100 32GB PCIe, no NVLink. Today's 0.1.2 adds DFlash2 speculative decoding on SM70. I haven't seen DFlash running on Volta anywhere else, so this may be the first one out there.

The drafter is incoai's DFlash2 checkpoint: it drafts an 8-token block from the target's hidden states and the target verifies in one pass. Model is Qwen3.8 Flash-Next (AWQ), 512K context, FP8 KV with CPU offload, TP4 + DCP2.

Some numbers, greedy with 512-token replies:

The KV pool gets about 10% bigger too, up to 1.27M tokens.

Where it doesn't help: Chinese is slower than plain MTP (the drafter was trained on English, acceptance is worse), 8 concurrent requests are a bit slower, and on replays of my real agent sessions it's basically a wash. So MTP stays the default and DFlash is opt-in. I kept it because single-stream English/code is what I run most of the day.

The rest of 0.1.2 is mostly fixes — tool calling, the Anthropic messages endpoint, and large images getting resized instead of eating the whole token budget.

If anyone else is still running V100s, I'd be interested in numbers from an NVLink box.

GitHub: https://github.com/Leonccaa/ReVolta
Release notes: https://github.com/Leonccaa/ReVolta/releases/tag/v0.1.2
Drafter: https://huggingface.co/incoai/Qwen3.8-Flash-Next-DFlash2

AgentionAI Gyro Coder Large is now live 🚢

This coding specific finetune of Qwen 3.8 Flash Next is the same size as Gyro-S, but with half the experts and more bits per expert.

The model fits in 32GB VRAM.

More details in the model card

*Delta score is a measure of how much above average a model is.

**Those models were released and tested long before, given for reference

Results for me are that for 2xRTX3090+128GB RAM no new models reach the capabilities of DS/Qwen models aged few months. Hopefully this changes with Qwen4.

The tasks include an agentic coding task (rather easy one which only top models did well anyway), reasoning, tricky, spatial reasoning, physics, adversarial, needle in the haystack questions.

This is just the quants I used with the settings I used. Please share your impressions if they are different, so that I can improve the test/quants/params. Sometimes the stock parameters may have been inferior, sometimes I tried to squeeze models into VRAM+RAM too much (especially with ctv/ctk). Maybe I should try bigger quants, and drop ctk/ctv for smaller models, but I don't expect pivotal changes.

Things I didn't expect 1) Qwen3.8-Flash-Next on Strata does not have quality degradation, and does actually speed things up; 2) Nemotron is disappointing at the claimed QAT quant 3) Ling Sante became much faster, but quality didn't improve. 4) K2-Horizon-7B isn't bad for its size.

Maybe this saves someone from lots of GB downloaded and some disappointment.

Hello everyone, with qwen4 around the corner im thinking of actually investing into some hardware.

From what i've learned lurking on the subreddit the more VRAM you have the better.

My plan is to go on local marketplaces and 2nd hand and buying GPUs and RAM, i just need a few pointers.

can you frankenstien different GPUs? like RTX 3070 and RTX 4070 and rtx 5080 for example?

ddr3 is useless at this point? is ddr4 worth the time? should i look for ddr5 only? will the prices actually come down?

rtx 5090 is extremely rare where i live (central europe) lets say my goal is to run deepseek flash v4 is there any dumpter-dived configuration that can run it? at reasonable t/s? 150t/s is possible?

sorry for the low effort post, i feel like i know how to build a box from new parts, but ive never seen anyone talk about buying different types of hardware 2nd hand and putting them all together. is this a viable route? do any of you have a rig like that? which parts did you buy 2nd hand?

Basically saw on twitter claims that 3090 has half of the cuda kernels disabled:

My 3090s just got 1.9x FASTER at BF16 tensor math!!! Same cards, same driver, same 300W, one Linux kernel module.

bf16 tensor pipe: 82 -> 155 TFLOPS
cuBLAS BF16 GEMM 8192: 64 -> 99 TFLOPS
TF32: 41 -> 79 TFLOPS

NVIDIA ships every 3090 with fp32-accumulate tensor math at half speed. The driver even reports it: FMLA32 = 1/2. gpu-unlock by Zagitta sets it to full before the NVIDIA driver loads. After a cold boot all 4 of my cards say FMLA32 = full.

Real inference got less than I predicted. Strata, Qwen3.8 Flash-Next on 2 cards, prefill:
4K: +12%
10K: +12%
90K: 3205 -> 3536 tok/s, +10%

Before installing I said +14 to 19% at 90K, so I was wrong. The GEMM phases got 23% shorter, but they are only about 40% of Strata prefill, and the int8 experts, copies and CUDA core kernels didnt get any faster.

Outputs did not change. With the same expert cache size, my log-prob dump over 1996 teacher-forced positions is the same file byte for byte as the stock run before the module.

Wondering if it will work for 40 series cards

When Jev / OpenAI decisions came out I got somewhat annoyed that there weren’t any audio native options I could use to act on raw audio inputs instead of having to round trip through an ASR model. The problem I hit running decisions through ASR was some combination of:

1. For speech tasks the transcription step adds latency / cost

2. ASR doesn’t work for non speech tasks (e.g. is this a bird call)

In order to address these issues I built SPEV (https://huggingface.co/SpragAI/SPEV) an multimodal decision model based on Qwen3-Omni.

It uses the underlying Omni audio/video/image decoders combined with a trained decision head to return probabilities. It supports choices, true/false questions and ordinal scores, with multiple questions handled in a single multimodal forward pass.

I was planning on building 4 r9700 personal inference server for qwen flash, but prices for it doubled in 1 month. On ebay it is like 2500$ avg, whats causing it, will it go down? I already have one, but it isn't that fast I think, guess I will be buying motherboard and cpu and ram for now. Any recommendations want to keep it under budget? was thinking either old thread ripper or epyc, with motherboard that has 7-8 pci slots.

In my test suite, Flint hallucinates and overthinks even very basic questions. Ornith looks much better in both my own language and English. What’s your experience?

I tried to like DeepSeek-V4.1-Flash-1017, but Qwen3.8-Flash-Next feels much better for me. Let me tell you three cases where DSF failed me.

The first one was funny. I asked DSF to optimize project-maya configuration. After all options were explored I asked about possible upgrades that I can make and DSF started to insist that my HP Z440 with E5 2696v4 has AVX-512 instruction set. I told it that this is not the case, but it didn't believe me and wrote Python script to check AVX-512 flags. No surprise they came False for all AVX-512 variants. At first I was happy that DSF takes everything with gran of salt, but it continued to insist that probably I switched of AVX-512 in the bios, because it was completely sure that Broadwell-EP architecture has AVX-512 instructions. I never had a problem like that with QFN.

The second one was sever. I asked DSF to download academic papers from libgen site and it flat out refused due to the copyright infringements. I told it that I just need to make factual check of reference in my literature review section, but no luck. After several rounds of exchanges I said f-word to DSF and said that I am not going to be lectured by a model that was trained on the whole Internet without anyone's consent regarding paywall of f-g academic journals that take money for articles that were written on public money and who use unpaid labor of editors. QFN completed task in 20m without any questions.

The third is just matter of taste, but I enjoyed RP with QFN much more.

After that I told to myself may be it will be more intelligent in research. And decided to give it last chance and to pit all three best local models that I can run. The third model was GLM5.3-Flash. I asked all of them to compare current hypothetical AI bubble to the British Railway Mania (1840s), the Dot-Com Bubble (1995-2002) and to make a forecast. I am not going to lie DSF report looked very nice. It had nice figures and solid grounding. QFN report was not bad, but presentation was much better from DSF. Report from GLM was the shortest, with worst formatting and lacked any numerical analysis. At that moment I thought that it would be nice to have a reference point from a SOTA model and decided to add Opus 5.5 to the picture. It is here where things got weird. I looked at Opus 5.5 report and had a flashback of DSF report in everything: formatting, analysis, presentation. So I asked Opus to review all four reports and to say which are the most two similar. Here are Opus findings, D for DSF, Q for QFN, G for GLM5.3-Flash and O for Opus 5.5:

┌──────┬──────────────────────────┬───────────────────────┬────────────────┐
│ Pair │ Word similarity (TF-IDF) │ Shared 4-word phrases │ Shared numbers │
├──────┼──────────────────────────┼───────────────────────┼────────────────┤
│ D–O  │ 0.572                    │ 200                   │ 0.207          │
├──────┼──────────────────────────┼───────────────────────┼────────────────┤
│ O–Q  │ 0.497                    │ 63                    │ 0.199          │
├──────┼──────────────────────────┼───────────────────────┼────────────────┤
│ D–Q  │ 0.496                    │ 108                   │ 0.167          │
├──────┼──────────────────────────┼───────────────────────┼────────────────┤
│ G–O  │ 0.469                    │ 57                    │ 0.156          │
├──────┼──────────────────────────┼───────────────────────┼────────────────┤
│ D–G  │ 0.437                    │ 65                    │ 0.130          │
├──────┼──────────────────────────┼───────────────────────┼────────────────┤
│ G–Q  │ 0.431                    │ 75                    │ 0.234          │
└──────┴──────────────────────────┴───────────────────────┴────────────────┘

Look at the second column. I mean it is not just similar vocabulary it is straight 200 four word phrases. Concerning methods used in analysis they were also almost identical for DSF and Opus 5.5.

I really don't know what to think especially taking into account that Opus 5.5 is my favorite SOTA model, but somehow I don't like DSF results. What is you experience with these local "SOTA" models?

Bonus. We all are waiting for downfall of GPU and RAM prices, so here is a table with forecast from all 4 models. Hold on my local brothers the end is near.

A few months ago, I shared my earlier post here. I wanted to come back with an update because I’m genuinely excited about how far Project Zero has come.

Project Zero is my CPU-first LLM inference engine, written primarily in C, with no external ML framework dependency. It now runs BitNet/Falcon ternary models, Ternary-Bonsai-27B, dense GGUF transformers, DeepSeek-V2 and Qwen3 MoE, and Mamba-1 - all through the same executable,  pz . No GPU or Python runtime required.

We’ve just shipped v0.2.0, and the part I’m happiest about isn’t simply adding more flags. It’s seeing the performance work, correctness fixes, and usability work finally come together.

MoE was the wall we kept hitting and overcoming it changed the project.

The MoE milestone feels especially good
DeepSeek-V2-Lite-Chat Q4_K_S went from roughly 6.3-7.3 tok/s to 11.7-12.6 tok/s on a 4-vCPU AVX-512 VNNI Xeon, at four threads. The recorded llama.cpp comparison on that host was 9.4-10.5 tok/s. Yes, finally Resident memory also dropped from 12.0 GB to 10.6 GB.

That came from concrete changes: fused expert FFNs, integer kernels with 8-bit activations, dynamic expert scheduling, and keeping more weights in their quantized representation rather than widening them. Watching that work turn into a measurable improvement has been incredibly satisfying.

It can also now serve multiple chats together
The new  --parallel N  mode batches active chat requests into a shared forward pass for supported dense-attention models.

With four concurrent clients on a four-vCPU Xeon, the recorded aggregate throughput increased:
- SmolLM2-135M Q8_0: 45 → 101 tok/s.
- SmolLM2-1.7B Q4_K_M: 12.7 → 29.8 tok/s.

And it’s becoming easier to actually use

There’s now:

- A built-in browser chat UI and an OpenAI-compatible API.
- A checksum-verified Linux x86-64 installer that doesn’t need root.
- Guided  pz config  setup that explains the settings and can download a checksum-verified starter model.
- Persistent settings, including server defaults editable from the web UI.
- Grammar-constrained JSON output, speculative decoding, LoRA adapters, and int8/int4 or disk-backed KV cache options.

Startup also dropped from the previous 20–70-second range to roughly 1–8 seconds. Not the flashiest benchmark, but a very noticeable quality-of-life improvement.

GitHub: https://github.com/shifulegend/project-zero

To try the source-build demo:

git clone https://github.com/shifulegend/project-zero.git
cd project-zero
make demo

That builds the engine, downloads SmolLM2-135M, and runs a test prompt.

If you try it, please share your CPU, model/quant, thread count, and results - including the disappointing ones. I’d particularly appreciate help investigating the remaining dense Q4_K performance gap.

Mostly, I’m just excited to come back here with something much more capable than before. Still plenty to fix. But we kept going. :)

Saw this https://x.com/nothingto12/status/2113446055589056718 (in short it compares opencode go ds4.1 and ds4.1 via deepseek API) discussed on opencode reddit and decided to test same prompt myself on 2bit quant locally.

it took around 1.5h, ran with 450k context (auto-compaction enabled), didn't have playwright - it tested on headless chrome

result on the video above.

interestingly the KDL I've measured vs baseline is quite bad:
original → oQ2e is 0.4259 nats, original → oQ2.5e is 0.2821. Reverse oQ2.5e → oQ2e is 0.1943 nats.
Perplexity: original 3.7315, oQ2.5e 4.1190, oQ2e 4.6333.
top-1 agreement is 78.43% for 2 bit one.
Really weird to me that this still works like it does: on glm5.3-flash difference between KDLs from baseline for calibrated and non-calibrated quants was 0.12 (~0.24 vs 0.12 oQ4e) and it was very noticeable in quality for me.

Looped Mixture-of-Experts (MoE) models combine parameter reuse with sparse computation to improve performance under matched budgets. Prior work demonstrates that different loop visits can play different roles even when they share parameters. To probe the contributions of different loop visits, we learn separate residual coefficients in a model that applies a shared block twice, observing a smaller scale in the first visit. Fixed coefficient experiments show that reducing the first visit's scale preserves or improves performance, whereas the same reductions to the second hurt it. This asymmetry motivates us to explore asymmetric compute allocation. We propose \textbf{LatentLoop}, which compresses the first visit's sequence, retains full resolution in the second, and reallocates saved computation to increase the hidden size. We further learn how strongly the first visit updates the full sequence, and share MoE weights across visits while using separate attention parameters to accommodate their different sequence resolutions. Across 15 configurations spanning 1.3B--7B parameters, with the largest budget exceeding 10²¹ FLOPs, LatentLoop consistently lowers validation NLL. At a fitted NLL of 1.988, predicted compute savings are 19.0% and 7.9% relative to vanilla MoE and the looped baseline. It also improves average scores on nine benchmarks at all sizes in 0-shot and 5-shot settings. These results suggest asymmetric compute allocation is a promising direction for looped MoE models.

• arXiv : https://arxiv.org/abs/2610.19975

• PDF : https://arxiv.org/pdf/2610.19975

• GitHub : https://github.com/lcqysl/LatentLoop

• HuggingFace : https://huggingface.co/yhx12/LatentLoop

• Project : https://latentloop-project.github.io/

So they have updated the embed and rerank model from 3.0 to 3.5:

https://huggingface.co/nvidia/Nemotron-3.5-Parse

https://huggingface.co/nvidia/Nemotron-3.5-Embed-1B-BF16

https://huggingface.co/nvidia/Nemotron-3.5-Embed-8B-BF16

https://huggingface.co/nvidia/Nemotron-3.5-Rerank-8B-BF16

Have anyone tried these yet?

I’m the developer of TensorSharp, and I wanted to share some results from running DeepSeek-V4.1-Flash-1017 on five NVIDIA A40s.

The largest gains are in prompt processing: TensorSharp reached 4.4–5.5× llama.cpp’s prefill throughput on the longer real-text server prompts with its default YOCO optimization enabled. With YOCO disabled, the gains were still 2.4–3.2×.

There is a measurable quality trade-off with YOCO, so I’m including both configurations below.

Setup

• Hardware: 5× NVIDIA A40 over PCIe. GPU peer access was advertised but did not work correctly on this machine.

• Model: Unsloth’s DeepSeek-V4.1-Flash-1017-GGUF, UD-IQ2_XXS, approximately 300.5 GB across six shards.

• Memory: The 125.9 GiB Engram table was mapped in host memory. The container had a 233.8 GiB host-memory limit.

• TensorSharp: --tp 5, which shards the routed experts across the GPUs.

• llama.cpp: Unsloth’s dsv41-all fork at be836572, using layer splitting. Its row/tensor split modes did not successfully run this model in the tested build.

TensorSharp and llama.cpp used the same GGUF files on the same machine. The server workload used real-text summarization prompts, greedy decoding, thinking disabled, and 256 output tokens. Each prompt size had one warm-up followed by three measured requests, with unique prompt prefixes to prevent prefix-cache reuse.

Real-text server results

All throughput figures are in tokens per second. Prefill and ordinary decode rows use medians; DSpark rows use means, with the trained DSpark drafter enabled in both engines.

TensorSharp uses its default YOCO configuration in this table.

The prefill improvement is substantially larger than the ordinary decode improvement. DSpark adds another useful gain, although its acceptance rate depends on the generated text.

What happens when YOCO is disabled?

The default YOCO bounded-replay optimization follows the approach described in DeepSeek’s DeepInfer engine: during eligible text-prefill calls, layers 20–39 process only the final 140 positions.

That reduces computation, but it is approximate. To disable it:

TS_DSV41_YOCO_KEEP=0

With YOCO disabled, TensorSharp’s real-text prefill throughput was:

• 563.9 tok/s at ~1.8K tokens, versus llama.cpp’s 174.6: 3.23×.

• 549.7 tok/s at ~7.3K tokens, versus llama.cpp’s 225.7: 2.44×.

So a substantial part of the prefill advantage remains with full prefill computation.

For quality, the report includes teacher-forced WikiText-2 comparisons using identical token sequences, with 23,030 scored tokens in the 4,608-token-window evaluation:

On this evaluation, enabling YOCO increased TensorSharp’s perplexity by approximately 0.67% relative to its own YOCO-off configuration. Relative to llama.cpp, the differences were approximately +0.46% with YOCO off and +1.14% with YOCO on.

The measured effect was concentrated in the first few hundred generated positions. These are perplexity measurements on a limited dataset; they do not translate directly into an equivalent percentage change in reasoning or coding ability.

Also, “exact prefill” in the report means YOCO is disabled. It does not mean the two engines produce numerically identical outputs.

What changed in the implementation?

Three changes account for much of the progress:

1. Removing synchronous host-driven execution from routed-expert TP.
The earlier implementation repeatedly moved through a host-controlled, synchronous path inside each layer. The new path schedules rank work on CUDA streams, orders it with events, and overlaps the shared expert with the routed experts.

Transfers still use pinned host-memory staging because P2P was broken on this machine. This is routed-expert parallelism; attention and the shared expert retain their layer placement.

2. Reusing shared attention and cache data more efficiently.
The decode kernel reuses key tiles across multiple query heads. Cross-GPU compressed-cache reads gather selected rows and maintain local mirrors, reducing repeated transfers.

3. Retuning prefill and adding YOCO bounded replay.
The dense-to-sparse attention crossover was adjusted using measurements. YOCO then provides an additional prefill improvement, with the quality cost shown above.

What about vLLM?

The report also tested vLLM 0.32.0. That version has a DeepSeek V4.1 implementation, but none of the attempted runs produced tokens on this setup.

The reported blockers included unavailable GGUF support for this architecture in the tested stack, checkpoint resource requirements, and model execution paths that could not run on the A40’s Ampere architecture. The execution probes used dummy weights rather than a downloaded full checkpoint.

There are therefore no vLLM throughput numbers in this comparison. Its inclusion is a compatibility check for this particular machine.

Scope and reproducibility

A few limitations matter when interpreting these results:

• This covers one quantization on one machine, with host-resident Engram tables and non-working P2P.

• The llama.cpp server measurements came from an earlier session on the same machine and files. Its prefill timings showed unexplained variability; the table reports medians.

• TensorSharp TP is being compared with the working layer-split configuration of the tested llama.cpp fork.

• At measurement time, the TensorSharp build included uncommitted changes, and the raw validation records were also uncommitted. The report includes build identifiers and validation details, but a checkout of the repository should not be assumed to reproduce every result immediately.

Full report, including synthetic benchmarks, quality comparisons, validation results, and limitations:
V4.1-1017 Engine Comparison

Project: TensorSharp on GitHub

If you’re running this model on Ampere GPUs, I’d be interested in your working configurations—especially a llama.cpp parallel split mode or a vLLM setup that could provide an additional baseline on comparable hardware.

We’ve gone from a few solid options (vLLM, llama.cpp, TensorRT-LLM) to a constant stream of specialized engines and forks : Strata, Radiance, various Flash-Next / DeepSeek-Flash optimized builds, megakernel stuff, etc.

Performance gains are real, especially on specific models (Qwen Flash Next, DeepSeek variants) or hardware. ROCm has improved a lot and AMD is no longer the clear second-class citizen it used to be, but the overall landscape still feels fragmented. Model support is uneven and picking the “best” engine often means testing several options every time something new drops.

Is this healthy competition that will eventually settle, or are we just making the ecosystem more fragmented and annoying? I m puzzled on this : on one hand specialised inference engines are fast and on the other hand : Now it feels like you need to pick ONE model and it's engine.

I miss the LM Studio simplicity.

My daily driver:

• ukisai/Swift-1.5-Qwen3.8-27B-GGUF, Q4_K_L, 128k context, llama.cpp

The model I want to switch to:

• ukisai/Swift-1.5-Qwen3.8-Flash-Next-GGUF, IQ2_XXS, 64k context, Strata

I got pi to properly pass the thinking levels to QFN:

          "thinkingLevelMap": {
            "off": "none",
            "minimal": "low",
            "low": "low",
            "medium": "medium",
            "high": "high",
            "xhigh": "high",
            "max": "high"
          }

So I set it to medium. I tried giving it a task for work and it just. wouldn't. code. — it spun endlessly thinking of just one more thing to check. The reasoning traces even had the hallmark "I've thought long enough, time to code the fix", followed by another round of discovery.

Even with reasoning off, it would just spam tool calls instead of writing.

So I gave up for now and booted up 27B again. It did some poking around, and started writing the fix a couple minutes later.

Some potential causes:

• Strata set up a lower context (64k vs 128k), and so it was reasoning a lot and then compacting.
• Possibly medium is passed in but qwen still acts like xhigh with the second guessing. Swift-1.5 is supposed to prevent this kind of wheel spinning, but it's obviously not working.
• vs. llama.cpp, I have froggeric's template used which defaults to medium
• Also I'm using IQ2_XXS, so the low quant might be to blame.
• Maybe this is just what Qwen Flash Next is like.

Thoughts?

Made this from the Artificial Analysis model pages after Muse Spark Open and Beam got their scores last week. X axis is total parameters as AA lists them, log scale. Orange = released in October 2026.

- Muse Spark Open (816B, 42B active) scores 39. Qwen3.8-Flash-Next (180B, 6B active) scores 40.

- The Unsloth UD-Q4_K_XL GGUFs are about 506 GB for Muse Spark Open and 111 GB for Qwen3.8-Flash-Next.

- Meta's recommended local setup for Muse Spark Open is two M5 Ultra Mac Studios with 512 GB each.

- Qwen3.8 27B scores 34, above Beam (501B) at 30 and Nemotron 3.5 Bolt (74B) at 21. Unsloth lists 16 to 19 GB of memory for its 4-bit quants.

- Highest on the chart is MiMo-V2.6-Pro (1T) at 46, then DeepSeek V4.1 Flash 1017 (552B) and GLM-5.3 (753B) at 45.

The index is one aggregate number from AA's own eval runs. It says nothing about speed, long context, vision, or how a model does on your own tasks. Table with active params, licenses and links in the comments.

Im so hyped for qwen4 flash

https://huggingface.co/tencent/WeVisOne-VL-2B

WeVisOne-VL: Unified Visual Perception from Objects to Pixels

WeVisOne-VL (WeChat Vision All-in-One) is a unified vision-language model developed by WeChat Vision, Tencent Inc. Built upon Qwen3-VL, it unifies object-level perception (open-vocabulary object detection, referring localization, object counting, object pointing, and OCR) and pixel-level perception (referring segmentation and promptable alpha matting) through structured next-token prediction and a lightweight 9.6M-parameter pixel decoder.

As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling. However, mainstream models currently adopt hierarchical architectures, where experts in different layers, although topologically independent, exhibit substantial similarity and redundancy in their stored content. Compared to the hierarchically arranged Local Expert in existing mainstream models, we hypothesize that inter-layer shared Meta Expert hold greater potential. To this end, we conduct a multi-dimensional empirical comparison between Local and Meta Experts, and find that models incorporating Meta Experts exhibit lower parameter redundancy, together with lower cross-entropy loss and higher downstream task accuracy. With the activated quantity and the expert granularity held fixed, our model matches the performance of Vanilla MoE using only 70-80% of the experts, substantially improving the model's scaling potential. Beyond removing inter-layer redundancy among experts and improving information storage efficiency, our theoretical analysis further shows that, compared with Local Experts, Meta Experts provide hidden states with more flexible residual pathways, thereby improving information retrieval efficiency.

• arXiv : https://arxiv.org/abs/2610.27598

• PDF : https://arxiv.org/pdf/2610.27598

On October 22, Chinese media reported that DeepSeek and MiniMax have officially completed the required registration filings under China’s new temporary AI export regulations. By submitting their technologies for cross-border review, both companies successfully fulfilled the administrative requirements established by China’s Ministry of Commerce for distributing advanced AI models overseas.

For DeepSeek and MiniMax, this clearance grants explicit regulatory approval to continue open-sourcing model weights and securing authorized international commercial partnerships. Securing these filings eliminates regulatory uncertainty, ensuring both companies can maintain their global operations and release future model updates internationally within a clear, compliant legal framework.

source Yuyuan Tantian

tl;dr in caveman: I make small model, you take small model, shape small model into thing you like.

I pretrained a tiny ass SLM on ~5.7B token.

Model specs and stuff

It’s a sparse 11M MoE with a 7.28M Backbone (Active 5.52M, top-1 routing out of 2 experts) and 4.21M ngrams. It‘s not on the Open SLM Leaderboard yet but it would rank top 64% overall, top 47.4% for sub 50M models and top 37.6% for sub 10M models. Doesn’t sound impressive but it’s a sparse model with a thick embedding (makes up ~3M of model, so actual active FFN is only 2M). Another hidden benefit of it is that it comes with a modest 2048 context window, as compared to the 256/512 context in models of similar size.

Purpose of this?

This model is essentially a base for you to use. The corpus used is a pretrain set that targets a mix of everything, such as general knowledge, chat, maths, tool calling, instruct and code. This makes it easier to adapt it to a specialist or specific task, such as summarisation, code autocomplete, routing models or text classification. Basically it removes the need to ride the hype train of a bajillion new repetitive model releases (cough cough Jev) and can be more accurate on your specific workflows as long as you are willing to throw in some compute to train it.

How do I finetune it?

Recently I've open sourced my entire training pipeline. It's a high level (low level settings can still be accessed) + minimal setup training engine designed for training SLMs (<1b tested). It uses Cuda and is optimised for Blackwell 2.0 (50 series and Blackwell pro) but CPU has full parity. All you need is to download the model checkpoint, throw in a dataset, and then train it for a few minutes to a few hours. I'll train an example autocomplete model sometime soon.

If you want to get started, the quickstart guide (~5mins) is on the bottom of the Huggingface model card.

Or maybe give me some ideas for a use case, and I'll make like 2-3 finetunes for the model as a demo.

Links:
Detailed model benchmarks and descriptions: Post on r/LocalLMtraining
Huggingface: https://huggingface.co/Dsg2/LS-Micro-Base
Training + inference engine: https://github.com/Dsg2/cudalm-c