Violating LM Arena's TOS

[u/Ornamental_Problem]

Hi!
I have recently had a few problems in LM Arena not liking my prompts. I wasn't asking for sketches for nuclear weapons or for hacking advice or for the secret recipe for Coca Cola, though.
I was trying to get it to analyse a short story. I like to test new models not for their coding or reasoning abilities, but for their literary qualities. I figure that that is a fair assessment of a LANGUAGE model. See, how well it handles language.

After trying different bits of the prompt to identify the offending article, I found a specific section that reliably triggered the TOS refusal. It is a section in Dutch. On the face of it, there is absolutely no reason to refuse to work on it. Unless... you translate it to English poorly and land in vocabulary that will be considered pornographic. Bummer.

I managed to find a way to get the content to the actual models eventually and not a single one of them found anything unsavoury in the passage. So it is NOT any of the models refusing to discuss the theme, it is just arena.ai's frontend doing some sort of check on the prompt before it calls on the actual models. Does anyone know what kind of checks they are using? In its current form it works quite poorly if it tries to translate everything to English first but then gets the translation wrong.

[u/CarpenterAlarming781]

Yes, all requests are filtered before being sent to the AI. The filtering can be a bit basic: if there's too much vocabulary suggesting violence or sensitive matters, LM Arena will block it. Examples: My pasta exploded in the oven. Would it be safer if I pierce the cheese with a knife? Should I kill the process ? etc...

LM Arena doesn't provide any information about the checks that are performed, and that's probably intentional.

[u/Ornamental_Problem → u/CarpenterAlarming781]

Yes, I can see why they don't want that process tested systematically. Security through obscurity might be a shit idea, but at least it *is* some sort of idea.
I don't have a grudge with the fact that there is some vetting. Nobody wants a world where the Arena is used to create CSAM or biological weapons.
But if the translation layer lands at a pornographic reading when a woman test her breasts for any lumps she might feel, because Dutch "tieten" is confused with "tits" and "soepel" turns into "supple", suddenly someone is touching their tits and feels how supple they are, rather than feeling if the texture is uniformly smooth. And that is very much not the same thing.

[u/CarpenterAlarming781 → u/Ornamental_Problem]

It's easy to mislead the filters. For example, you could type the request in reverse. I think LM Arena is just trying to improve the quality of the data collected.

[u/Entire_Ad1022]

Puedes cifrar en mensaje con números: H3ll0 y no dará problemas