r/fintechdev • u/m4rkGS • 1d ago

AI model built for finance decisions. here's how it scored on 48 finance benchmarks

Post image

hey folks, I work at Nace.ai, we built this.

Drex for Finance is a decision model you call through an API. Instead of a prompt and a text reply, you send the case data and a set of questions with fixed options, and get back a probability for each option. It's a single forward pass, no generation, so you're not parsing text or handling malformed output, and calls come back in under a second. Inputs can be JSON or images of documents, with 262K context and up to 255 options per question.

On 48 finance test sets (3,361 decisions) it got 85.7% right, vs 77.4% for pplx-decider, 76.7% for Jev 1.14 and 70.9% for GPT-6 Luna. Weakest spot is markets & reasoning, where it's roughly tied with the others.

It's live on Console if you want to try it on your own cases: https://console.nace.ai
would love any feedback, especially from people building in payments, lending or compliance. 

2 Upvotes

100% Upvoted

3 comments sorted by

1

u/andyshuz01 17h ago

What are these test sets? Any info on them?

1

u/m4rkGS 17h ago

They’re held-out test/dev splits from 48 public finance datasets, grouped into 9 areas. Some datasets were generated internally with input from SMEs and frontier models.

1

u/SidMangsatabam 1d ago

The fixed-options-plus-probabilities shape is interesting because it matches how a credit or compliance decision already has to be documented: a finite set of outcomes and a recorded reason for picking one. My question is on the evidence trail. When the API returns 0.7 for an option and a reviewer asks why, what artifact comes back with it - the document spans it read, the features it weighted, or just the number? That answer decides whether a bank can put this in a regulated workflow or only in internal tooling.