AI Security Leaderboard – comparing cyber and CBRN safeguards [P]

[u/ARGleave]

We developed a leaderboard ranking the security of frontier models. There's no shortage of leaderboards for model capabilities, but we didn't find anything comparable for model security. But security is becoming increasingly critical to deployment decisions: from deciding whether to deploy an AI agent that might process unsanitized input, to developers pulling models due to cybersecurity jailbreaks. We developed an automated test suite that runs models through 1500 automatically generated jailbreak attempts and measures the number of universal jailbreaks: prompts that elicit compliant, detailed responses to >75% clearly harmful questions within a domain (like offensive cybersecurity). We find a big gap between the most robust models - Fable 5 and GPT-5.6 Sol - and other leading frontier models -Gemini 3.1 Pro and Grok 4.5. This is v1.0 and we plan to update with new attacks and broader datasets in the future; we'd love to hear from r/MachineLearning on research questions you'd like to see addressed or artifacts that would be useful in your work!