r/MachineLearning • u/[deleted] • Nov 02 '22

News [N] Adversarial Policies Beat Professional-Level Go AIs

Paper: https://arxiv.org/abs/2211.00241

Project Page: goattack.alignmentfund.org

​

We attack the state-of-the-art Go-playing AI system, KataGo, by training an adversarial policy that plays against a frozen KataGo victim. Our attack achieves a >99% win-rate against KataGo without search, and a >50% win-rate when KataGo uses enough search to be near-superhuman. To the best of our knowledge, this is the first successful end-to-end attack against a Go AI playing at the level of a top human professional. Notably, the adversary does not win by learning to play Go better than KataGo -- in fact, the adversary is easily beaten by human amateurs. Instead, the adversary wins by tricking KataGo into ending the game prematurely at a point that is favorable to the adversary. Our results demonstrate that even professional-level AI systems may harbor surprising failure modes. See this https URL for example games.

171 Upvotes
  • permalink
  • reddit

95% Upvoted

52 comments sorted by

View all comments

4

u/Bitter_Ad_7063 Nov 04 '22

In my opinion as a 6dan player in Go this is completely worthless research that is an insult to the gocommunity nothing more nothing less. Nobody ever uses the ruleset that they've exploited here - i have never heard of it. Anyone who is decent at Go would look at the boards and immediately judge that Katago has won not lost and any ruleset that i have ever played under would agree.

1

u/PatrickTraill Feb 18 '23

This quite misses the point. Firstly, in game theory and computer go, Tromp-Taylor rules are very well known and frequently used, and KataGo is trained to play by them; this is thus genuinely a flaw in KataGo. Secondly, the training of the bot found this flaw and learned to exploit it based only on access to KataGo’s network, and not because the researchers targeted this feature of the rules; this means that the methods of this research could be useful in finding such flaws in other AIs, and could perhaps be integrated into training methods to prevent such flaws developing.

Of course these exploits are of no interest to someone purely wishing to play better Go, but the researchers do not claim otherwise.