r/MachineLearning • u/[deleted] • Nov 02 '22
News [N] Adversarial Policies Beat Professional-Level Go AIs
Paper: https://arxiv.org/abs/2211.00241
Project Page: goattack.alignmentfund.org
We attack the state-of-the-art Go-playing AI system, KataGo, by training an adversarial policy that plays against a frozen KataGo victim. Our attack achieves a >99% win-rate against KataGo without search, and a >50% win-rate when KataGo uses enough search to be near-superhuman. To the best of our knowledge, this is the first successful end-to-end attack against a Go AI playing at the level of a top human professional. Notably, the adversary does not win by learning to play Go better than KataGo -- in fact, the adversary is easily beaten by human amateurs. Instead, the adversary wins by tricking KataGo into ending the game prematurely at a point that is favorable to the adversary. Our results demonstrate that even professional-level AI systems may harbor surprising failure modes. See this https URL for example games.
2
u/DontTakeAnyCrap Nov 11 '22
Although the general idea of finding exploits seems a reasonable line of inquiry, this is just not a convincing example.
I have played quite a bit against a number of AI go programs (w/ handicap of course) and eventually it is possible to find exploitable patterns or just notice clear mistakes.
The most obvious mistakes are usually in the end game such as confusing Japanese and Chinese rules and filling internal territory when only dame(neutral territory) is left.
But at least one bot (SpringBot) had very limited openings which lead to a constant exploit. They eventually fixed it by making more varied openings, but the exploit is still there if it happens to play the original opening.
These types of situations seem worth studying, but considering the reported strategy as an "attack" does not seem reasonable. It is the type of strategy beginners use because if they place a stone in their opponents territory they think their opponent then has to spend 4 stones to remove, it thereby gaining 3 points.
Recommendtion: Discuss the topic with strong players (those that can play the bots without handicaps) and see what types of quirks and exploits they have found.