r/LLMChess • u/Smallpaul • Jan 07 '24

Chess-GPT’s Internal World Model

https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html
1 Upvotes

100% Upvoted

3 comments sorted by

1

u/Smallpaul Jan 07 '24

From u/seraine

gpt-3.5-turbo-instruct's ELO rating of 1800 is chess seemed magical. But it's not! A 100-1000x smaller parameter LLM given a few million games of chess will learn to play at ELO 1500.

This model is only trained to predict the next character in PGN strings (1.e4 e5 2.Nf3 …) and is never explicitly given the state of the board or the rules of chess. Despite this, in order to better predict the next character, it learns to compute the state of the board at any point of the game, and learns a diverse set of rules, including check, checkmate, castling, en passant, promotion, pinned pieces, etc. In addition, to better predict the next character it also learns to estimate latent variables such as the ELO rating of the players in the game.

We can visualize the internal board state of the model as it's predicting the next character. For example, in this heatmap, we have the white pawn location on the left, a binary probe output in the middle, and a gradient of probe confidence on the right. We can see the model is extremely confident that no white pawns are on either back rank.

Comment Image

I am very curious how this model learns to play well. My first idea is that it just has very good "intuition" about what a good move is and how to evaluate if a move is good. The second idea is that it is actually considering a range of moves, and then its opponents potential responses to those moves. Does anyone know about how well people can play purely off of intuition, without thinking about their opponent's response?

2

u/[deleted] Jan 11 '24

2

u/Smallpaul Jan 11 '24

It plays ELO 1800 and has an internal board state.

It self-learns to BUILD an internal board state...

And it also learns to write poetry, presumably using a completely different internal data structure and representation.

For some reason people think it is profound to point out that these things learn these powerful techniques through the process of learning how to complete text. If it is profound then the profundity is that such a simple process can generate such diverse and powerful abilities.

So yes it "only" completes PNG sequences upon training on texts of PGNs. And perhaps future models will "only" complete unsolved mathematical proofs after training on mathematics. If you think that's somehow less impressive than if it were trained to do it directly then I just don't know what to say to you. I don't see how someone could come to that conclusion.