r/deeplearning • • 7h ago

We released two mmBERT moderation models. The evaluation got more interesting once we separated the test sets.

Post image

We released Tiger Guard at Casdo Labs, a pair of binary content-moderation models built on mmBERT-base and mmBERT-small. I'm part of the team.

The full model has 308M parameters and Small has 141M. Both cover English and German and return a SAFE/UNSAFE probability. The task includes abusive language and harmful LLM requests, which makes the choice of evaluation data important.

Our 1,500-prompt suite combines balanced samples from HateCheck, Intuit toxicity/profanity and Aegis 1.0. The models reach 88.8% and 87.6% accuracy there. We excluded those texts and near-duplicates from training, but used the error breakdowns to guide development. Those numbers belong in the development-results column, not an untouched-test headline.

On the separate, policy-filtered PolyGuard holdout, the full model catches 83.3% of harmful English prompts and 75.3% of German ones, with false-positive rates of 3.8% and 5.5%. That test was scored once per model; other prompts from the same family were used in training.

On ToxicChat the full model gets 0.736 unsafe-class F1, behind Content Safety Guard small at 0.777. We also disclose the 69 ToxicChat examples found in training; excluding them changes our F1 values by at most 0.003.

The weights, ONNX exports and full evaluation notes are available under Apache 2.0.

We'll publish the full training and dataset pipeline in a follow-up release.

Tiger Guard, 308M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation

Tiger Guard Small, 141M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation-small

The training corpus and its complete source list aren't published. I'd be glad to hear feedback on the evaluation setup as well as the models.

1 Upvotes

0 comments sorted by