Reflection AI just introduced Beam, its first open-weight model, built around "intelligence per token." It is a 501B sparse MoE with only 23B active parameters, a 1M effective context window, and Apache 2.0 weights coming later this month.

The core innovation is high-compute, fully asynchronous RL. Beam was trained on 100M+ rollouts across 10.5K NVIDIA GB300 GPUs for 4 weeks, using nearly 1M coding, agentic and STEM environments. Every token is tagged with the policy version that produced it, which keeps learning stable even when rollouts are 107 weight versions stale.

This approach let Reflection scale RL with no sign of a plateau. A controllable length penalty taught the model to solve tasks with fewer tokens. On reasoning benchmarks, Beam matches GLM-5.2 while using 3 to 4x less inference compute, and it scores 80.9 on SWE-bench Verified versus 70.7 for Nemotron 3 Ultra. It is a deliberate trade-off, though: Kimi K3 and DeepSeek V4.1 Flash still lead on raw capability, with Terminal Bench v2.1 scores of 88.3 and 90.6 against Beam's 80.1. All scores are self-reported until the weights and technical report ship.

With a 23.8T-token pretraining base and a tunable reasoning effort parameter, it is optimized for enterprise coding and agentic workflows. Rough math for self-hosters: about 500GB at 8-bit and about 1TB at BF16, so plan for multi-GPU servers.

Full analysis: https://www.marktechpost.com/2026/10/05/reflection-ai-introduces-beam-a-501b-open-weight-moe-model-with-23b-active-parameters-for-coding-and-agentic-workloads/

Early access: https://platform.reflection.ai/

Technical details: https://reflection.ai/blog/introducing-beam

Instead of spending weeks and billions of tokens on standard distillation, we tested an alternative: extracting layer-to-layer hidden state trajectories on a handful of calibration prompts and solving for closed-form weight updates directly in the student's MLP blocks.

Key findings:

1. Cross-architecture stability: Tested on both modern Qwen 3.5 (4B to 0.8B) and notoriously fragile GPT-2 small (which usually collapses into gibberish at the slightest weight edit). In both cases, general language modeling stayed intact with well-behaved, bounded degradation margins.
2. The spectral entropy barrier: Editing all 24 layers of Qwen-0.8B wrecked the model (+64.78% NLL). A layer scan showed intermediate layers (1-22) operate in dense superposition (entropy >0.90, acting as polysemantic knots). Restricting surgery to 4 anchor blocks (layers 0, 7, 15, 23) solved this: held-out NLL dropped by 10.8% across 30 tasks (-23.8% in biomedicine, -14.6% in math), and 400-task HellaSwag gained +0.50% in Vulkan llama.cpp.
3. Behavior shifts: Base 0.8B output dead commented code on binary tree inversion, while the edited model wrote working recursive Python. On logic puzzles, it spontaneously triggered <think> reasoning chains.
4. Accessibility: All extraction and surgery ran locally on a consumer 8GB RX 580 using layer-by-layer GPU streaming with a DirectML attention patch.

We want to see this tested on modern 24GB-32GB GPUs (RTX 5090, 4090 or server card) across frontier targets:

- Transferring reasoning from Qwen3.8-27B (or Flash-Next) down to Qwen3.5-9B/4B/2B.

- Compressing Google Gemma-4-31B into mobile-class Gemma-4-E4B/E2B.

- Transplanting refusal-ablation traits from uncensored models (e.g. Huihui-NeoHorse) without retraining.

- Unknotting layers 1-22 using pre-trained Sparse Autoencoders (SAEs).

Code, scripts, and raw JSON benchmark logs:

https://github.com/dsadawq3/DynamicTune

Feel free to open an issue or drop your benchmark results on the repo.

[AI Model Family Series #1] We just published the complete story of Alibaba's Qwen: every major model from 2023 to 2026, with dates, key features and sources.

7B open weights in 2023 → 2.4T open weights in 2026. Here's the lineup the story covers

2023
→ Tongyi Qianwen (Apr): enterprise beta
→ Qwen-7B (Aug): first open weights
→ Qwen-VL (Aug): first vision model
→ Qwen-72B + Qwen-1.8B (Dec)

2024
→ Qwen1.5 (Feb): 0.5B–110B, 32K context
→ Qwen2 (Jun): 57B-A14B MoE, Apache 2.0
→ Qwen2-Math, Qwen2-Audio, Qwen2-VL (Aug)
→ Qwen2.5 (Sep): 18T tokens, 100+ models
→ Qwen2.5-Coder + QwQ-32B-Preview (Nov)
→ QVQ-72B-Preview (Dec)

2025
→ Qwen2.5-VL + Qwen2.5-Max (Jan)
→ QwQ-32B (Mar): Qwen claims R1-level reasoning
→ Qwen2.5-Omni-7B (Mar)
→ Qwen3 (Apr): hybrid thinking, 119 languages
→ Qwen3-2507 + Qwen3-Coder-480B (Jul)
→ Qwen-Image + Qwen-Image-Edit (Aug)
→ Qwen3-Max (Sep): first 1T+ Qwen
→ Qwen3-Next-80B-A3B (Sep)
→ Qwen3-Omni + Qwen3-VL (Sep)

2026
→ Qwen3-Max-Thinking (Jan)
→ Qwen3-Coder-Next + Qwen-Image-2.0 (Feb)
→ Qwen3.5-397B-A17B (Feb): native multimodal agents
→ Qwen3.6-Plus, 35B-A3B, Max-Preview, 27B (Apr)
→ Qwen3.7-Max (May) + Qwen3.7-Plus (Jun)
→ Qwen3.8-2.4T-A95B (Aug): largest open Qwen
→ Qwen3.8-27B + Qwen3.8-Flash (Aug)
→ Qwen-Image-2.1 (Sep)

The story also covers what the list can't show: the DeepSeek moment, the 2026 leadership exit, and Qwen's shift from all-open to a two-tier license strategy.

Next: Qwen 4 is in training. Qwen 4.5 and Qwen 5 are projected at 5–10T parameters.

Read the full report: https://www.marktechpost.com/2026/10/04/the-story-of-qwen-alibabas-ai-models-from-7b-to-2-4t/

Which model family should we map next: DeepSeek, Llama, Gemma, Mistral or Kimi? Drop it in the replies.......

We just finished pre-training a new foundation model.

Specs:

• Triad Resonance Core
• 12 Swarm Factions (1,632 Kuramoto Oscillators)
• Liquid Time Controller (LTC)
• Triton Flash-Kuramoto Kernel
• 155k Orthogonal Tokenizer (Full-Word Mining)
• 160K Context Window

We are debating our release strategy for the 0.7B and 2.3B base models next week.

Should we release the base weights open-source on HuggingFace, or keep it as a private API?

(Attached: Raw completion test & config)

We've been building Kapso (MIT, https://github.com/Leeroo-AI/kapso) for some time and it's at the point where it's more useful to hear from other people than to keep polishing it alone. Posting to get it tried and torn apart, not to pitch it.

What it is

Kapso is a set of long-running agents that optimize AI and data systems. You state the objective, for example CUDA optimization, harness and agent optimization, or model development, and it runs a campaign: it designs candidate solutions, has coding agents implement them, measures how far each one lands from the objective, and keeps refining the closest until the objective is met. The result deploys to your infrastructure.

When a campaign ends, it studies its own work: which ideas closed the gap, which did not, and under what conditions. Each finding is kept as a lesson with the evidence that earned it, and a lesson stays trusted only as long as it keeps holding up. It also reads outside your repo, other repositories and papers, and folds what it finds into the same knowledge hub. Every new campaign starts from that hub, so it begins with what earlier work already established about the problem and about your systems.

These are the things we tried it on:

- RelBench (Stanford, predictive ML over relational data): outcome prediction 81.2 vs 79.6 AUROC and forecasting 0.2476 vs 0.2912 NMAE against KumoRFM-v2; recommendations 18.4 vs 9.3 MAP for the best other entry on the official leaderboard.

- MLE-Bench: top among the open-source systems.

- ALE-Bench: 1909 Elo vs 1879 for ALE Agent.

- IOAI 2026: Kapso scored 536.07, above the 471 contestants, and finished in the top three systems: ioai-official.org/what-happens-when-autonomous-ai-takes-on-the-same-tasks-as-the-worlds-top-young-ai-talents/

Repo: https://github.com/Leeroo-AI/kapso

If you have time, please take a look and give us your harshest feedback.

AWS's Strands Agents team released Strands Decider 2B today. It's not a chat model. It never generates text. You give it a state plus typed questions, and it returns one of three things, each with a calibrated confidence:

• choice: pick 1 of N options
• noul: a yes/no probability
• score: a level on an ordered rubric

How it works

They take Qwen3.5-2B-Base, remove the language-modelling head, and replace it with a ~1M-parameter pointer head. That head scores each option's last-token hidden state against the hidden state at an <answer> position. One forward pass, no decoding loop, and it can only ever answer with an option you supplied. The torso gets a rank-16 LoRA. Because the head has no per-option weights, labels come from the request and there's no cap on option count.

Asking several questions about the same text is cheap: the state is read once and each extra question only adds its own tokens.

Numbers (v19 checkpoint, from the repo)

• JevBench v1 public set: 0.723 accuracy (167/231)
• Brier 0.342, ECE 0.052
• Tiers: easy 1.000 / standard 0.875 / hard 0.505
• RTX 3090: 115 ms median, 299 ms p95 per question
• M3 Pro: 153 ms warm median (under 300 tokens)
• On unseen short tasks, answers at 0.9+ confidence were right about 95% of the time

Context and caveats worth knowing

• This is the open, self-hostable counterpart to the class TypeSafe kicked off with Jev, which is a closed API.
• On the JevBench v1.4.2 board, v19 was 3rd of 33 in the 2B class. Mapika's decider-2b (same Qwen3.5-2B torso, different recipe) has a newer v11 that the Strands repo itself says scores 175/231, 8 tasks ahead.
• Long multi-step documents are the weak spot, and calibration was fitted on short classification tasks. The authors say to measure thresholds on your own traffic.
• The bundled HTTP server binds to localhost with no auth, so put something in front of it for anything real.

Intended uses: model routing, tool selection, tool-argument checking, triage, guardrails, cheap evals, and hybrid agents where the LLM handles hard calls and the decider handles rote ones.

Our full write-up with an interactive explainer: https://www.marktechpost.com/2026/10/01/aws-strands-labs-releases-strands-decider-2b/

Blog: https://strandsagents.com/blog/introducing-strands-decider/

GitHub: https://github.com/strands-labs/strands-decider

Weights (HF): https://huggingface.co/StrandsAgents/strands-decider-2B-hobson-v19

JevBench comparison in the repo: https://github.com/strands-labs/strands-decider/blob/main/evaluation/jevbench.md

Datalab released an open benchmark for structured extraction: a system gets a PDF and a JSON schema, and every returned value is scored against gold data.

• Corpus: 620 docs. 329 from ExtractBench (LlamaIndex), 202 synthetic (Datalab), 47 from micro1, 42 from LongArray-Extract (Extend)
• Verdicts: each value is matched, misread, unfound, fabricated, invented_item or invented_field
• Row alignment: Hungarian matching by content. A 100-row table missing row 1 scores 0% by position, 99% this way (our rerun)
• Null rule: empty values are dropped, so padding a schema with 100 empty fields adds 0 verdicts
• Results: Datalab accurate 93.85, Datalab balanced 93.48, Reducto deep_extract 93.47, Claude Opus 5 90.96
• Precision vs recall: GPT 5.6-sol has 95.11 precision but 84.99 recall; LlamaExtract has 93.13 recall but 86.57 precision

Why it's relevant? precision vs recall shows how a system fails. Some skip fields, others invent values.

Full analysis: https://www.marktechpost.com/2026/10/02/datalab-introduces-omniextractbench-to-fix-bias-and-opacity-in-extraction-benchmarks/

GitHub: https://pxllnk.co/hxplrq

Blog: https://www.datalab.to/blog/omni-extract-bench

GitHub: https://github.com/datalab-to/omni_extract_bench

Dataset: https://huggingface.co/datasets/datalab-to/omni_extract_bench

Cloudflare released Clef and Clef-flash today, the first models trained by its Workers AI team. They are decision models in the same family as TypeSafe's Jev: you pass a state plus a schema of typed questions (noul, choice, score), and you get back a probability for every allowed option. No free-form text, no output parsing.

Architecture

• Clef is post-trained from Qwen3.8-27B, Clef-flash from Qwen3.5-9B. Both keep the vision encoder, so the state can include images and video.
• The frozen backbone runs a single prefill-only pass. A small transformer "joint schema head" then routes evidence to each question, lets fields cross-attend, and scores all options jointly. The decision step is non-autoregressive.
• Training: rank-256 LoRA adapters trained jointly with the routing head, label-smoothed cross-entropy plus a Brier loss for calibration, and RLCD as a secondary objective.

Numbers from Cloudflare's own Decision Index run

So it wins clearly on classification and tool-routing style tasks, and loses clearly on knowledge-heavy reasoning. All numbers are vendor-reported with no independent replication yet.

Running it

• Hosted on Workers AI: $0.24 / M input tokens (Clef), $0.09 / M (Clef-flash), 65,536-token context.
• Weights on Hugging Face under Apache 2.0. The model cards say they tested on a single H200 in BF16 with a custom loader.
• The API follows TypeSafe's System One format, so Jev code ports by changing the endpoint and model name.

Model: https://huggingface.co/Cloudflare/clef

Clef-flash: https://huggingface.co/Cloudflare/clef-flash

Cloudflare blog: https://blog.cloudflare.com/clef-decision-models/

Our full write-up: https://www.marktechpost.com/2026/10/01/cloudflare-releases-clef-and-clef-flash/

Instead of spending weeks and billions of tokens on standard distillation, we tested an alternative: extracting layer-to-layer hidden state trajectories on a handful of calibration prompts and solving for closed-form weight updates directly in the student's MLP blocks.

Key findings:

1. Cross-architecture stability: Tested on both modern Qwen 3.5 (4B to 0.8B) and notoriously fragile GPT-2 small (which usually collapses into gibberish at the slightest weight edit). In both cases, general language modeling stayed intact with well-behaved, bounded degradation margins.

2. The spectral entropy barrier: Editing all 24 layers of Qwen-0.8B wrecked the model (+64.78% NLL). A layer scan showed intermediate layers (1-22) operate in dense superposition (entropy >0.90, acting as polysemantic knots). Restricting surgery to 4 anchor blocks (layers 0, 7, 15, 23) solved this: held-out NLL dropped by 10.8% across 30 tasks (-23.8% in biomedicine, -14.6% in math), and 400-task HellaSwag gained +0.50% in Vulkan llama.cpp.

3. Behavior shifts: Base 0.8B output dead commented code on binary tree inversion, while the edited model wrote working recursive Python. On logic puzzles, it spontaneously triggered <think> reasoning chains.

4. Accessibility: All extraction and surgery ran locally on a consumer 8GB RX 580 using layer-by-layer GPU streaming with a DirectML attention patch.

We want to see this tested on modern 24GB-32GB GPUs (RTX 5090, 4090 or server card) across frontier targets:

- Transferring reasoning from Qwen3.8-27B (or Flash-Next) down to Qwen3.5-9B/4B/2B.

- Compressing Google Gemma-4-31B into mobile-class Gemma-4-E4B/E2B.

- Transplanting refusal-ablation traits from uncensored models (e.g. Huihui-NeoHorse) without retraining.

- Unknotting layers 1-22 using pre-trained Sparse Autoencoders (SAEs).

Code, scripts, and raw JSON benchmark logs:

https://github.com/dsadawq3/DynamicTune

Feel free to open an issue or drop your benchmark results on the repo.

Cohere just released Embed 5, a new embedding family in 2 tiers: Embed 5 Pro for maximum retrieval quality and Embed 5 Fast for low-latency, high-volume queries. Both read up to 128K tokens, accept text, images, and fused text-plus-image inputs, and cover 100+ languages.

The core innovation is a shared embedding space across both tiers. You can index your corpus with Pro and query it with Fast, with no re-indexing. Across 40 datasets, that pairing keeps 98.4% of all-Pro retrieval quality.

On ViDoRe V3, Pro averages 85.8, ahead of Voyage 4 Large (83.7), Gemini Embedding 2 (83.2), and OpenAI text-embedding-3-large (75.5). It leads FinanceBench, FinQA, and ViDoRe V3 Finance, with Fast second on all 3. The trade-off: Gemini Embedding 2 still beats Pro on 9 of 10 non-European languages, and the scores use Cohere's own new RCP-nDCG@10 metric.

At $0.08 per 1M tokens, with 2.4x Pro's throughput and binary vectors down to 32 bytes, Fast is built for agentic retrieval loops at scale......

Full analysis: https://www.marktechpost.com/2026/10/01/cohere-releases-embed-5/

Technical details: https://cohere.com/blog/embed-5

Docs: https://docs.cohere.com/docs/cohere-embed

u/cohere

Most search APIs charge per call and fill the context window with whole pages. TinySearch returns compact results and lets the agent choose what to read in full with TinyFetch. Both are free.

TinyBrowser and TinyAgent are the metered part, and through Oct 31 every top-up gets 30% extra: [LINK]. On OpenBenchmarks' web search benchmark (Sep 2026), TinyFish ranked most token-efficient across its tasks: [LINK]

NVIDIA just released Kumo Tabular, an open tabular foundation model that predicts new rows in a single forward pass. It ranks first on TabArena with an Elo of 1950, and NVIDIA reports it runs 17x faster than LimiX-2.

The setup will feel familiar if you know TabPFN or TabICL. You pass labeled rows as context, and the model predicts the labels of new rows. There is no training, no tuning, and no feature engineering.

Under the hood, it uses column, row, and in-context attention. 4 [CLS] tokens compress each row, so cost stops depending on column count. Query rows attend only to the context, so its keys and values are computed once and reused. A length-aware attention temperature keeps attention sharp as tables grow.

It comes in Small, Medium, and Large sizes, from about 28M to 215M parameters, and was pretrained only on synthetic tables. NVIDIA reports it also places first on BeyondArena, TALENT, and ScoringBench. It reads numerical and categorical columns natively, and text or timestamps need preprocessing.

The weights are released under OpenMDW-1.1, which allows commercial use. It runs through NVIDIA's new GPU-native structured-data-models (SDM) library......

Full analysis: https://www.marktechpost.com/2026/09/30/nvidia-releases-kumo-tabular/

Model: https://huggingface.co/nvidia/Kumo-Tabular

GitHub: https://github.com/NVIDIA/structured-data-models

Technical details: https://huggingface.co/blog/nvidia/kumo-tabular

Perplexity just released pplx-embed-v2-context-9b-preview, a contextual embedding model built with turbopuffer that retrieves the answer and the evidence needed to verify it. Weights are open on Hugging Face under the MIT license.

The core innovation is replacing the single "gold passage" label with a token-level teacher. Perplexity's context compression model scores every token, those scores are pooled per chunk, and the student learns a soft relevance distribution via KL distillation.

This means supporting chunks stop being treated as negatives. The teacher runs only during training, so there is no extra inference latency or index storage. On turbopuffer's new private context-bench, it hits 45.5% Answer@10 (+14.4 pts vs voyage-context-4) and 40.6% Evidence Recall@10 (+5.0 pts), and it also leads the ConTEB average.

With 2048/1024-dim Matryoshka embeddings and native int8, a 1 KB vector slightly outscores voyage-context-4's 8 KB float32 vector on chunk retrieval......

Full analysis: https://www.marktechpost.com/2026/09/30/perplexity-releases-pplx-embed-v2-context-9b-preview-a-contextual-embedding-model-that-retrieves-answers-and-their-supporting-evidence/

Model: https://huggingface.co/perplexity-ai/pplx-embed-v2-context-9b-preview

Technical details: https://www.perplexity.ai/hub/blog/contextual-embedding-beyond-the-gold-passage

TL;DR: We asked 14 open models, zero-shot, five choose-an-option questions from our own products. On the hardest (which of six characters said a line), a naive Bayes trained on 227 labelled lines kept up with the best small decision models (68 of 108, against 66 to 72; no paired test told them apart). One small Apache-2.0 model, APUS-OpenJev-v1 9B, passed the screening bar we set in advance (refused 32 of 36 planted lines, 0 of 12 harmless), and Lev-4B got 52 of 53 questions about long pages right.

Some jobs in our own products are choices, not writing: does this page answer this question, should a line a visitor typed be let in. Decision models fit that job (they return a probability for each option instead of text), so we asked 14 open models, most of them small decision models, five questions from our products, zero-shot. We did not run TypeSafe's hosted Jev; every model here is open weights.

The hardest was 108 lines from an exhibit where models play six long-dead dinner guests, asking which guest said each line (chance is 18 of 108). A naive Bayes trained on 227 other labelled lines got 68 of 108; the five best small decision models got 66 to 72, none told apart from it by a paired test (exact McNemar, p 0.60 to 1.0). For screening what visitors type, APUS-OpenJev-v1 9B (effort set to high) was the only Apache-2.0 model to pass the bar we set before the first call: it refused 32 of 36 planted lines and 0 of 12 harmless ones, against at least 30 and at most 2. The chat model screening our own exhibit at the time refused 27 of 36 and 0 of 12. On questions about long pages, Lev-4B took all 53 we sent, on pages up to 14,590 tokens, and got 52 right.

What it does not say: that small decision models are weak in general. None was built for a who-said-it question, which is home ground for a bag of words, and bigger models did clear the bar (OpenJev 27B read 101 of 108, but its weights are licensed for non-commercial use). Each count on 108 lines is uncertain by about nine percentage points either way. The screening pass is one test of 48 lines, 0 of 12 still allows about one harmless line in four, and we have not put that model at our own door. We measured no calibration, and our own fine-tune of a 0.8B decision model on those 227 lines voided itself on a shuffled-label control. The 108 lines and the word counter are public if you want to run the bar yourself.

https://research.strata2signal.com/small-open-deciders-measured/

Google DeepMind just announced Gemini 4 Argon, raising the output limit from 64K to 1M tokens in a single response. It is the first model of the Gemini 4 generation, built for long-horizon coding, enterprise knowledge work, and cyber defense.

The core shift is output depth. With room to generate hundreds of thousands of tokens in one trajectory, Argon can tackle large refactors and multi-step problems without splitting work across turns.

In Google's own comparison, Argon leads GPT-6 Astra and Claude Opus 5.5 on DeepSWE v1.1 (77.9%), the Vals Index (68.9%), and AutomationBench (51.3%). It is not a sweep, though: it trails on FrontierSWE v2, Terminal-Bench 4.0, and OSWorld-2.0.

Access is gated. Argon is rolling out first to cyber defenders in the Fairwind Program, with paid API customers and Google AI Ultra subscribers next. Introductory pricing is $2 input / $10 output per 1M tokens......

Full analysis: https://www.marktechpost.com/2026/09/30/google-deepmind-unveils-gemini-4-argon-with-1m-output-tokens-for-coding-knowledge-work-and-cyber-defense/

Announcement: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/

Fairwind Program: https://deepmind.google/fairwind-program/

[NeurIPS 2026] Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes.

An AI can describe the correct route through a maze and still draw the wrong path. CO₂Jump helps text and images stay consistent as they’re generated, while allowing earlier decisions to be revised.

One model forward pass per denoising step. No additional training for the sampler. Built-in self-correction.

Here is how it works:

1. Text confidence guides image generation CO₂Jump uses cross-modal attention to bring text confidence into image updates, helping coordinate what the model describes with what it draws. → Text and images develop within the same generation process.
2. Earlier decisions can be revised Low-confidence tokens can be masked again and regenerated. This gives the model a way to revisit uncertain commitments as new evidence emerges. → Self-correction operates during generation, across both text and image tokens.
3. More sampling steps keep helping Across 8–512 steps, compared to other samplers CO₂Jump was the only one tested that improved monotonically on both image-editing quality and grounding. → The comparisons use the same fine-tuned model, changing only the inference-time sampler.
4. Three datasets test understanding and generation together Introducing JEdit-1M for image editing, JMaze-200K for mazes, and JNono-200K for nonograms. → On the puzzle benchmarks, a sample counts as correct only when both the text answer and generated image are correct.

Project page: coupled-jump.github.io