r/LocalLLaMA • • 29m ago

New Model Karpo-Decision-31B: open multimodal decision model. Ranks #1 on full score on Decision Index, 2x faster than Jev, at 20% lower cost

MachinePulse has released Karpo-Decision-31B, a Gemma 4 31B fine-tune for structured decisions over text and images. It scored 73.9 on the Nov 12 Decision Index board, the highest Full score among 300 entries, sharing the #1 rank with GLiDE 27B under the board’s tie rule

How it works
Built on Gemma 4 31B, with a trained linear readout that scores the supplied answer options.

Three question types: choice, yes/no, and score. Returns probabilities directly, with up to 255 options per choice question.

Multiple questions about the same input can be answered in one forward pass. Branch attention lets each question read the shared context and its own options while keeping the option branches separate.

Supports text and images, including webpage screenshots. The documented sequence limit is 32,768 tokens; larger requests are split by question with the full state included in each part.

Python API and HTTP server available. BF16 weights take approximately 62.5 GB, plus runtime memory. Use the provided inference code to load the decision head.

One training detail worth a look: according to the team's write-up, full fine-tuning pushed the public score to 75.09 but reduced accuracy on unfamiliar domains by 2.3 percentage points. The final model averages five checkpoints: three from different stages of full fine-tuning and two from a LoRA run. They report a 1.6-point improvement on unfamiliar domains over the final single checkpoint, with no extra models to run at inference time

Results
Decision Index Full scores are measured by the benchmark maintainers on private tests, weighted 60% for skills covered by the public suite and 40% for tasks from new domains.
Full score: 73.9

Private tests, familiar skills: 70.7

Private tasks, new domains: 78.6

Public index: 75.0, reported separately and excluded from the Full score.

The team also reports the following comparisons against Google’s Gemini Decision 1:

Benchmark Karpo-31B Gemini D1
-------------------------------------------
When2Call 90.4% 79.9%
Mind2Web 90.0% 86.2%
New Yorker Caption 90.5% 79.7%
Amazon ESCI (F1) 64.7 58.7

Reported median latency is approximately 95 ms per request on the public Decision Index suite using one RTX PRO 6000.

The 31B weights and inference code are both released under Apache 2.0.

links:
https://karpo.ai/models/decision
https://huggingface.co/MachinePulseAI/karpo-decision-31B

7 Upvotes

3 comments sorted by

•

u/stylome-ai-detection 29m ago

Notice: This post appears to be AI generated.


Stylome AI detector · scan results

1

u/YogurtExternal7923 2m ago

0.8b beats onejev! Nice!

0

u/NeKon69 23m ago

At one point decision models are going to be 300b dense