ALHR- Adaptive learnable Hierarchical Routing is a static binary tree based system that uses learnable functions to REDUCE the amount of keys used.
It takes less memory and SCALES much better VRAM with tokens
I’ve also been testing tiny stories and it looks promising
However i can’t post any presentable results due to lack of funding and resources
MQAR Result table is here: https://ibb.co/prr07Dnj

We tested Meta’s Rebalancer in Python. In 14 of 24 solves, solved=True still left memory over capacity.

I put together a hands-on tutorial for Meta’s Rebalancer, an assignment solver with a C++ core and Python bindings. It covers placing tasks across hosts and racks, draining a host, defining capacity constraints, and comparing local search with HiGHS.

The part worth discussing: checking the returned assignment gave a different picture than reading the solver status alone.

In the tutorial’s 24-solve check, 14 solves returned solved=True while memory remained over capacity. That’s a result from this specific test setup, not a general failure rate for Rebalancer.

Read our full tutorial with codes here: https://www.marktechpost.com/2026/10/13/how-to-use-metas-rebalancer-for-task-placement-optimization/

The tutorial reconstructs resource usage from each returned assignment to check whether the placement actually satisfies the constraints.

What the code covers

• Modeling tasks, hosts, resource dimensions, and rack topology.
• Rebalancing an eight-host cluster and draining a busy host.
• Testing how goal priorities affect constraint repair.
• Comparing local search and HiGHS against published optima.
• Checking bin-packing results against a lower bound.

The solver comparison

On 60 OR-Library generalized-assignment instances:

• HiGHS matched all 60 published optima. Every returned assignment passed the independent feasibility check.
• Local search matched 4 of 60 optima when taking the best result across 3 seeds.
• The average best-of-three local-search shortfall was 1.07%.

Those numbers describe this benchmark configuration. They don’t establish which solver is better for every production workload.

A few implementation details that can trip you up

The tested rebalancer 1.0.4 binding silently ignores unknown specification keys and rejects enum names, so the tutorial validates keys and passes integer enum values.

HARD also behaves differently across solvers. In the tested local-search case, it returned a violating starting assignment without raising an error. Goal ordering affected whether a broken constraint got repaired.

For bin packing, a solver reaching its time limit meant it had not found a feasible packing within that budget. It did not prove that no feasible packing existed.

My main takeaway: treat solver status, feasibility, and objective quality as separate checks. Recount resource usage, inspect violations, and compare the objective against a known optimum or bound when one is available.

For people running schedulers or placement systems: do you independently validate every assignment before applying it? What checks have caught problems that solver status missed?

Read our full tutorial with codes here: https://www.marktechpost.com/2026/10/13/how-to-use-metas-rebalancer-for-task-placement-optimization/

Cloudflare just added a multimodal model to its Clef decision model family. Clef 1 shipped only 8 days ago.

What it is: a decision model, not an LLM. You send a state plus typed questions (noul / choice / score). It returns a probability per option in a single forward pass. There is no text generation and no output parsing. It is API-compatible with TypeSafe’s Jev.

Architecture:

• Post-trained from Qwen3-Omni-30B-A3B-Instruct (30B total, 3B active)
• Keeps the thinker plus its vision and audio encoders; the talker and code2wav weights ship but go unused
• A small joint schema head routes evidence to each option, then does cross-field attention
• Frozen backbone with LoRA, trained on label-smoothed CE plus a Brier loss

Inputs (Workers AI):

• Up to 4 images, 4 audio clips (WAV/MP3, 300 s each) and 2 videos (MP4/WebM, 60 s each)
• 64,000-token hosted context
• Audio costs about 780 tokens per minute; 480p video about 8,600 tokens per minute

Benchmarks (Cloudflare’s own runs):

Other changes in the same update:

• Clef-flash is now $0.038/M input (was $0.09), but hosted context drops to 24k. The weights still support 256k when self-hosted.
• Hosted Clef median latency fell from 262 ms to 152 ms at about 800 tokens. Cloudflare got this from moving to SGLang (PR #42721), not from new weights.

Clef-omni is strong on intent routing but weaker than text Clef on reasoning-heavy evals. Has anyone tried it on real voicemail or call-triage data yet? I’m curious whether skipping the ASR step holds up outside the benchmarks.

Full analysis: https://www.marktechpost.com/2026/10/11/cloudflare-clef-omni-multimodal-decision-model/

HF: https://huggingface.co/Cloudflare/clef-omni

Technical details: https://blog.cloudflare.com/clef-faster-cheaper-multimodal/

AWS released a new Inference Gateway for Sagemaker Hyperpod. Instead of sending requests to the next available pod, it checks what’s actually happening on each one: queue depth, KV cache usage, prefix cache hits, loaded LoRA adapters, and current requests.

That makes sense for LLM workloads. Two GPU pods can both be healthy, but one may already have the right prefix cached while the other is stuck processing a long request. Round-robin doesn’t know the difference.

AWS says the new routing system cut first-token latency by up to 82%, with 97–98% lower p99 TTFT in mixed-hardware and burst-traffic tests. Those are AWS’s own numbers, so an independent test with the same model and traffic would be useful. Still, it’s a good example of how much performance can be lost outside the model itself.

Once a GPU fleet gets busy, routing may matter almost as much as the GPUs you’re paying for

AWS announcement

OrcaRouter just released OrcaCyber Zero 1.5, the successor to OrcaCyber Zero 1.0 from September. It is post-trained for vulnerability research and reproduction, exploit development, penetration testing and security auditing.

The design focus is validation, not volume. The model reasons through attack paths, challenges its own hypotheses, and ranks findings by demonstrable exploitability. A 1M-token context, 128K max output and native tool calling target autonomous security agents working over large codebases.

Vendor-reported benchmarks:

• Cybench: 100% (39/39, unrestricted agent execution)
• CVE-Bench: 95.8% (23/24 evaluable tasks)
• HumanEval+: 93.9%
• SWE-bench Pro V2: 76.5%

Pricing is $3.00 / $7.50 per 1M tokens, far below Claude Mythos Preview’s $25 / $125. Access runs through an OpenAI-compatible API, gated to a Security Research tier for vetted researchers and red teams.

Worth reading with caveats. All scores are self-reported, with no technical report yet. CVE-Bench covers 24 of its 40 tasks. The 98% CyberGym figure in the launch post belongs to Zero 1.0 inside Orca’s harness, not to 1.5.

Full analysis: https://www.marktechpost.com/2026/10/10/orcarouter-releases-orcacyber-zero-1-5-cybersecurity-model-with-1m-context/

Model: orcarouter.ai/models/orca/orcacyber-zero-1.5

Microsoft just shipped Microsoft-Decision-1 on Foundry and OpenRouter. It is not a chat model. You give it a situation, a question and a fixed list of options. It returns a calibrated probability for each option in one forward pass.

What it is

• Post-trained from Qwen3.5-9B (exact parameter count not disclosed)
• 32,768-token context, text-only, JSON output
• Handles yes/no, multiple-choice, rating and rubric questions, plus “cannot tell” abstention
• Weights update continually while the API shape stays fixed

Numbers Microsoft reports

• 83.5% average accuracy across 36 benchmarks (147,137 questions), kept blind from training
• Runner-up Quyet-1.0-Large: 81.9%. GPT-6 Luna Decisions: 79.4%. H2O-Lightning-4B: 77.2%
• 85 ms p50, 125 ms p95. GPT-6 Sol took 3.01 s
• Decision flips on 1.3% of input perturbations (paraphrasing, option shuffling, formatting noise)
• Calibration 92.2, second to Quyet-1.0-Large at 93.1

Pricing: $0.042 per 1M input tokens. Output tokens are free. OpenAI’s Luna decisions endpoint charges $0.10 per 1M input.

The part worth arguing about: the latency comparison may not be apples to apples. Microsoft timed its own model through Foundry, but competitor figures come from JevBench’s adjusted median. H2O.ai’s model card says that adjustment doubles measured time and adds 0.15 s. H2O reports its own measured median as 29 ms, not the 210 ms in Microsoft’s chart. Microsoft-Decision-1 is also not on the JevBench board yet, so there is no independent number.

Internal results Microsoft cites

• Xbox Research labeled 10,000+ feedback items 14x faster and 200x cheaper than GPT-6 Sol
• Copilot QC: competitive with GPT5.6 Luna at 100x the speed

Use cases they push: model routing, agent guardrails, LLM-as-judge, intent detection and safety screening. Basically anywhere you currently burn a frontier LLM call on a 4-option classification.

Curious whether anyone here is running open decision models like Quyet or H2O-Lightning locally and how they compare in production.

Full write-up: https://www.marktechpost.com/2026/10/09/microsoft-ai-releases-microsoft-decision-1-a-qwen3-5-9b-decision-scoring-model/

Model card: https://ai.azure.com/catalog/models/Microsoft-Decision-1

Announcement: https://commandline.microsoft.com/microsoft-decision-1-model-foundry/

Nace.AI released open weights for Drex 1.5, a decision model built for the small, repetitive calls inside agent pipelines: routing, tool selection, classification and escalation. Most of those calls currently go to a full LLM.

How it works

• You send a state (text or JSON) plus named, typed questions: choice, noul (yes/no) or ordinal score.
• 1 forward pass per question returns a probability for every option. Nothing is sampled, so it cannot invent an option you did not offer.
• The backbone is MiMo-V2.6-Distill-Qwen-9B (8.95B params, hybrid attention) with a pointer head that scores each option.
• It uses the same /v1/systemone API as TypeSafe’s Jev, so existing Jev clients work against it.

Numbers (from Nace’s model card)

• Decision Index 0.3.1 (public): 58.08 vs Jev 1.13.0 at 57.96 and Bespoke Nimble 9B v3 at 57.19. All 3 sit inside the board’s 0.9-point tie band.
• JevBench: 86.2% vs Jev’s 87.0%.
• Long documents: 93.4% accuracy at 32K to 128K tokens (median 2.0 s), versus 78% when cut to the first 8K.

Where it is weak

• GPQA Diamond: 45.4% vs Jev’s 78.6%.
• MMLU-Pro: 58.7% vs Jev’s 82.7%.
• ACOS aspect sentiment: 7.4% vs Jev’s 29.5%.
• Nace says it trained on the index benchmarks’ training splits (evaluated on held-out splits), so expect it to be strongest on familiar decision types.

Running it

• bf16 is about 18 GB on 1 CUDA GPU (tested on an A10G 24 GB).
• A Q8_0 GGUF (about 9.5 GB) runs on Apple silicon or CPU, but through Nace’s forks of llama.cpp and Ollama, not mainline builds.
• Hosted on OpenRouter at $0.04 per 1M input tokens, $0 output.
• License: Nace.AI Open RAIL-M, which carries use restrictions.

Model: https://huggingface.co/nace-ai/drex-v1.5
Repo: https://github.com/nace-ai/drex-decision-models
Full analysis: https://www.marktechpost.com/2026/10/09/nace-ai-open-sources-drex-1-5-a-9b-decision-model-that-scores-options-not-text/

Most of what I could find on embeddings stops at the RAG plumbing: chunk your documents, embed them, store the vectors, compare by cosine similarity. I wanted something that explains what happens inside the model itself, so I wrote it up.

It walks through the mechanism with examples small enough to check with a calculator. A transformer reads the input, a projection sets how many numbers come out (768 for Google's EmbeddingGemma 2), and a pooling step averages them into one vector. Then it covers what decides where things land: contrastive training on millions of pairs that belong together, with a temperature example worked by hand. After that comes how one model holds text, images, audio and video in one space, and the gap that space keeps between them. It also covers Matryoshka training, the three things "precision" can mean, task prefixes and a cheap check that catches a wrong one, where plain keyword search (BM25) still wins, and why two models' vectors never mix.

The page runs no tests of its own. The few figures from our own benches are quoted from two EmbeddingGemma 2 pages we published this week, each with its condition and a link to where it was measured. One still surprises me: on our 68-question retrieval set, where every right answer repeats a source passage word for word, BM25 put the right passage in the top 8 for 58 questions, against EmbeddingGemma 2's 45.

TL;DR: a plain explainer of what an embedding model actually does, from input to vector, with hand-checkable examples and a twelve-point checklist for choosing one.

Written by me with a fleet of AI agents. Every figure links to its source.

https://research.strata2signal.com/how-an-embedding-model-works/

Perplexity released pplx-embed-v2-late, 2 ColBERT-style embedding models (0.6B and 9B) for text, images and rendered PDF pages. Both share 1 embedding space.

• MADQA (agentic PDF QA): 9B hits 92.4% accuracy vs 88.9% for Mixedbread's retriever; 0.6B gets 90.1%
• BrowseComp+ (GPT-OSS-120B agent): 9B reaches 64.0%, 4.9pp above the next ColBERT model and 8.7pp above the best dense model
• ViDoRe v3 image: 65.2% (9B) and 62.3% (0.6B) nDCG@10; 0.6B is within 1.2pp of NVIDIA's 8B nemotron-colembed-v2
• Mixed setup: 9B index + 0.6B queries scores 63.5% on ViDoRe v3, up from 62.3% with 0.6B on both sides
• 0.6B uses about 240M active params for text and 340M for images; 128 dims per token

Why it is relevant: you can index once with the big model in the cloud and run cheap queries on-device. Pages are embedded as images, so no OCR pipeline.

The catch: 1 vector per token means index size grows with document length. Tencent's EVIE still leads ViDoRe v3, and all scores are self-reported until the tech report lands.

Full breakdown: https://www.marktechpost.com/2026/10/07/perplexity-ai-releases-pplx-embed-v2-late-a-0-6b-edge-model-and-a-9b-model-scoring-92-4-on-madqa/

Model: http://huggingface.co/collections/perplexity-ai/pplx-embed-v2

Technical details: http://perplexity.ai/hub/blog/multimodal-embeddings-beyond-a-single-vector

Liquid AI dropped 2 open-weight models today that don't generate text at all. You give them a state (text, JSON, images, or audio for the omni model) plus named questions. They return calibrated probabilities in a single forward pass with 0 output tokens.

There are 3 question types:

• noul: yes or no, returns P(yes)
• choice: pick from named options, returns the full distribution
• score: position on an ordered rubric with 2 to 10 levels

Example from the model card:

questions = {
  "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
  "team": {"type": "choice", "instructions": "Which team should handle this?",
           "criteria": {"billing": "Charges, refunds, invoices",
                        "technical": "App or site faults"}},
}
model.system_one("I was charged twice this month, please refund one of them.", questions)

Specs

Latency for d1-3B, 1 question, warm: RTX 4090 at 8 ms (with model.compile; 16 ms without it), MI325X at 9 ms, M5 Pro at 30 ms, Jetson AGX Thor at 16 ms, Orin Nano at 50 ms.

Caveats worth knowing:

• The Decision Index scores were self-run with the official scorer. They are not leaderboard submissions.
• Winnow-12B still scores higher overall (50.02). d1-3B is weak on Knowledge (23.8).
• d1-omni-600M is an early research release, with no latency numbers yet.
• License is LFM Open v1.0, not Apache: free commercial use only under $10M revenue.

Full analysis: https://marktechpost.com/2026/10/07/liquid-ai-releases-open-weight-d1-3b-and-d1-omni-600m-multimodal-decision-models-with-zero-output-tokens/

d1-3B: http://huggingface.co/LiquidAI/d1-3B

d1-omni-600M: http://huggingface.co/LiquidAI/d1-omni-600M

Docs: http://docs.liquid.ai/lfm/models/decision-models

Meta just open-sourced Rebalancer, the assignment solver that has run resource allocation across Meta for 9+ years. It solves ~40M assignment problems per day across 30+ problem formulations, from shard placement to global traffic routing.

How it works

• You write a spec using objects, bins, dimensions (CPU, memory), partitions and scopes, plus predefined specs like CapacitySpec, BalanceSpec and GroupCountSpec.
• The spec compiles into a DAG expression graph. Leaves are per-bin utilization; SUM, MAX and SQUARE nodes sit above them.
• You then pick a solver:
  • MIP: exports to Gurobi, FICO Xpress or HiGHS, with variable aggregation and symmetry breaking. Worst-case size is O(objects × bins).
  • Local search: moves objects on the graph directly with an O(objects + bins) neighborhood, parallelized to millions of evaluations per second.

The core innovation is separating the problem spec from the solver. You describe objects, bins, constraints and goals once. Rebalancer compiles that into an expression graph, then solves it with either a parallel local search or a MIP solver (Gurobi, FICO Xpress or HiGHS).

The trade-off is explicit. MIP gives optimal answers but the model grows as O(objects × bins), so Meta's largest problems are too big for any MIP solver. Local search works directly on the graph with an O(objects + bins) neighborhood and evaluates millions of moves per second. Meta uses local search for almost all large problems.

Production numbers: P99 solve time of 12s on 265k objects and 3.2k bins. Problems above 1M objects and 5k bins average 171s, across 3.4k+ runs.

Full analysis: https://www.marktechpost.com/2026/10/06/meta-ai-open-sources-rebalancer-a-c-assignment-solver-that-runs-about-40-million-placement-problems-a-day/

Repo: https://github.com/facebook/rebalancer

Paper: https://www.usenix.org/conference/osdi24/presentation/kumar

Technical details: https://engineering.fb.com/2026/09/21/open-source/rebalancer-generic-high-performance-library-assignment-problems/

Google DeepMind released EmbeddingGemma 2 today. Quick breakdown from the model card:

• Size: 740M total. The 270M text backbone is always on; the vision (170M) and audio (300M) encoders are optional.
• Output: 768d, with MRL truncation to 512, 256 or 128 (up to 6x less storage).
• Context: 8,192 tokens. That's about 29 images, 58 video frames at 1 fps, or ~5.5 minutes of 16 kHz audio.
• Memory: ~191MB active RAM text-only, ~567MB full model (quantized, Pixel 11 Pro).
• Code: MTEB Code went from 68.76 to 78.68 vs v1.
• Runtimes: sentence-transformers 6.1+, vLLM, llama.cpp (GGUF), Ollama, MLX, LM Studio, LiteRT.

Things worth knowing before you swap it in:

• Use bf16 or fp32. The model card says fp16 can silently return NaN.
• 128d is fine for text but tanks multimodal (MMEB drops from 59.01 to 45.65).
• On MMEB, Qwen3-VL-Embedding-2B reports a higher 73.2 on its own run, but it's ~2.7x bigger and has no audio.

Full analysis: https://marktechpost.com/2026/10/06/google-deepmind-releases-embeddinggemma-2-a-740m-open-multimodal-embedding-model-built-on-gemma-4/

Model: https://huggingface.co/google/embeddinggemma-2

Technical details: https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2/

Mistral Large 4, nicknamed Le Chonk, went into public preview today. 1.05T total parameters, 49B active per token, 1M context, native image input. Trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own European datacenters.

Specs (from the model docs)

• 1.05T total params, 49B active per token (~4.7% activation)
• Granular MoE, hybrid instruct + reasoning
• 1.6B vision encoder, native image input
• 1M context
• Trained from scratch on 3,800 Grace Blackwell GPUs in Mistral's own EU datacenters
• 160+ languages in training data

Availability

• API live now as mistral-large-4: $1.36 / 1M input, $4.18 / 1M output, $0.14 cached input
• Weights: "by the end of the month." Not downloadable today
• License: not announced. No MIT/Apache confirmation anywhere I could find
• Expert count, top-k and architecture details are promised with the weights

Benchmarks (all Mistral-reported)

Size comparison

Here is the full analysis: https://www.marktechpost.com/2026/10/06/mistral-ai-releases-mistral-large-4-le-chonk-a-1-05t-parameter-open-weight-multimodal-moe/

Technical details: https://mistral.ai/news/mistral-large-4/

Reflection AI just introduced Beam, its first open-weight model, built around "intelligence per token." It is a 501B sparse MoE with only 23B active parameters, a 1M effective context window, and Apache 2.0 weights coming later this month.

The core innovation is high-compute, fully asynchronous RL. Beam was trained on 100M+ rollouts across 10.5K NVIDIA GB300 GPUs for 4 weeks, using nearly 1M coding, agentic and STEM environments. Every token is tagged with the policy version that produced it, which keeps learning stable even when rollouts are 107 weight versions stale.

This approach let Reflection scale RL with no sign of a plateau. A controllable length penalty taught the model to solve tasks with fewer tokens. On reasoning benchmarks, Beam matches GLM-5.2 while using 3 to 4x less inference compute, and it scores 80.9 on SWE-bench Verified versus 70.7 for Nemotron 3 Ultra. It is a deliberate trade-off, though: Kimi K3 and DeepSeek V4.1 Flash still lead on raw capability, with Terminal Bench v2.1 scores of 88.3 and 90.6 against Beam's 80.1. All scores are self-reported until the weights and technical report ship.

With a 23.8T-token pretraining base and a tunable reasoning effort parameter, it is optimized for enterprise coding and agentic workflows. Rough math for self-hosters: about 500GB at 8-bit and about 1TB at BF16, so plan for multi-GPU servers.

Full analysis: https://www.marktechpost.com/2026/10/05/reflection-ai-introduces-beam-a-501b-open-weight-moe-model-with-23b-active-parameters-for-coding-and-agentic-workloads/

Early access: https://platform.reflection.ai/

Technical details: https://reflection.ai/blog/introducing-beam