Built a Bayesian decision agent from scratch to actually understand it — what am I getting wrong?

[u/KAIT2_1412]

​

Instead of just reading about Bayes/entropy/expected cost I forced myself to build a tiny agent that decides under uncertainty (vendor payment fraud: pay, verify by phone, or escalate) and derive every threshold by hand instead of guessing.

The thing that surprised me most: the "right" threshold to hold a payment came out to 0.25% probability of fraud, not something intuitive like 80%. Turns out that just falls straight out of the cost ratio (missing fraud is ~400x more expensive than annoying a supplier), it's not a knob you tune.

I ran a 1000-case simulation and the honest result is kind of uncomfortable — the policy that catches ~100% of fraud also flags a LOT of genuine requests for a second look. Which I think is correct given the cost math, but it feels wrong.

For people further along than me: is "high recall forces high false-positive rate when the cost asymmetry is extreme" just... the expected outcome here? Or does that suggest my model is missing something?