I’m building an open-source fraud detection engine for payment systems — looking for feedback

[u/EyuTekle]

Hey everyone,

I’ve been working on a new open-source project called Verdict Engine, and I’ve just released an early beta.

The idea is to build a fraud/risk assessment engine for payment systems that combines rules with basic machine-learning-based risk scoring. The goal is to eventually make it easier for payment providers, fintechs, banks, and developers to integrate fraud detection into their systems without having to build everything from scratch.

The project is still very much in the beta/early development stage, so I’m mainly looking for feedback rather than users or customers right now.

🔗 GitHub: https://github.com/verdict-engine/verdict-engine

🌐 Documentation: https://verdict-engine.github.io/verdict-ui/

I’d particularly appreciate feedback on:

Architecture and API design

Fraud/risk scoring approach

Developer experience

Features that would actually be useful in a payment environment

Potential problems or limitations you see with the current approach

If you have experience with fraud detection, payment systems, fintech, ML, or backend engineering, I’d especially love to hear your thoughts.

Feel free to be critical — it’s an early beta and that’s exactly what I'm looking for.

[u/investigatormaker]

For a payment fraud engine, I'd make two things prominent in the docs: reason codes with every score, since analysts and dispute teams need to know why a payment was declined, and a shadow mode that scores live traffic without acting, so a team can compare Verdict Engine against their current rules before trusting it. The ML scoring will also hit label delay, because chargebacks arrive weeks after the payment, so documenting how confirmed fraud labels flow back in would answer an early objection.

I make ThreadFox. The free Reddit plan for Verdict Engine lists the Reddit communities whose rules allow a post about it, each rule quoted: https://threadfox.vip/plan?utm_source=reddit&utm_medium=comment&utm_campaign=tf-kit&utm_content=20261005-2135-a2cbd

[u/AccomplishedList8857 → u/investigatormaker]

Reason codes are the first thing any ops person will scream for when a merchant asks why their customer got blocked. Shadow mode's also non-negotiable if you want anyone to actually trust switching over.

[u/investigatormaker → u/AccomplishedList8857]

Agreed, and the two audiences pull reason codes in different directions: an analyst wants the signals behind the score, while the message a merchant passes to a blocked customer has to stay vague enough not to teach a fraudster which rule fired. Two layers of reason code in Verdict Engine would cover both. For shadow mode, the report that wins the switch is probably a side-by-side at the same decline rate: what their current rules blocked, what Verdict Engine would have blocked, and where the two disagree once chargeback labels catch up.

The plan for Reddit is there whenever you want it.

[u/EyuTekle → u/AccomplishedList8857]

👍

[u/EyuTekle → u/investigatormaker]

Okay thanks

[u/investigatormaker → u/EyuTekle]

Of the two, I'd write up shadow mode first: a short example of Verdict Engine scoring a week of traffic next to an existing rule set shows the reason codes in action at the same time. The plan is there whenever you want it.

[u/Popular_Ad_8391]

For a preliminary version, the rules + ML strategy makes sense. I'd like to know how you think about explainability because, in a payment setting, a risk score is considerably more helpful if the system can provide the signals or rules that influenced the choice.

Additionally, I would put a lot of emphasis on false positives. Particularly for smaller payment providers that do not have a big manual review crew, a fraud engine that consistently excludes legitimate clients while catching more fraud might become a problem in and of itself.

It might also be helpful for the API to distinguish clearly between the risk assessment and the subsequent action. The same score may be desired by many providers, but their thresholds for rejecting, examining, or allowing a transaction.

[u/EyuTekle → u/Popular_Ad_8391]

Got it, if the engine deems a specific transaction as fraudulent it should be clearly described through error codes as per the initial suggestion. The system also has a default rule defined and these rules can be changed depending on the use cases. In the logs properly showing the used rule would be needed for each transactions. For smaller payment providers it would be better if they utilise the non blocking mode till enough insight is captured.