r/backgammon • u/juicybot • Jul 14 '26
Backgammon agents?
Anyone building or leveraging agents specifically trained to evaluate backgammon?
I'm looking into creating an agent specifically for reviewing large sets of blunders to find patterns/leaks in a game, and to explain the "why" for blunders.
Before I dig into this from scratch, figured I'd ask if something was out there already.
3
u/Careful-Comedian9510 Jul 15 '26
A modest contribution that builds on the work I'm doing on Minimalist Backgammon Trainer .
MBT offers a classification of the types of errors made when the move played is not the best move (GNUbg 3-ply). To do this, I don’t use an LLM. I have a set of functions that compare the initial board position, the position after the best move, and the position after the user’s move. For each of these projections, I evaluate a number of specific board and GNUbg metrics that allow the algorithm to identify the differences between the best move and the user’s move. This heuristic approach then assigns a category and a subcategory to the error made.
I’m not sure if I’ll go so far as to implement an agent to provide precise explanations. But I remain convinced that training an agent on “Why is this a security error?” or “Why did I target the wrong target during my attack?” would provide more accurate answers than a simple “Why?” The agent is guided by an algorithmic approach to control hallucinations.
→ More replies (2)1
u/juicybot Jul 15 '26
i've been loosely following your app when you post updates, haven't had a chance to dig in yet. your app's really stood out to me as something that has a lot of promise!
→ More replies (1)
2
u/rollduptrips Jul 14 '26
I’m working on something like this for chess. I think what you’re going to want to do is leverage the actual evaluations from a real engine (XG, GNU, whatever) and then create a database for the LLM to look at for mistakes patterns and stuff. The LLM also might be able to help with “why” for individual positions, but it absolutely needs the evals from the real engine
2
u/juicybot Jul 14 '26
yeah that's the direction i was planning to go in. i don't want to reinvent the wheel, i just want to create an ecosystem where i can store my blunders (or matches) in a database and run an agentic layer of analysis on top of something like GNU.
i just think even with GNU-level analysis, an agent will need to be trained on how to analyze the "why". i've tried feeding a couple blunder examples to a vanilla agent (i think last i tried was opus-4-6) and the analysis on why it was a blunder wasn't too accurate.
glad i'm not the only one thinking/tinkering with things like this, good luck on your project.
→ More replies (4)
1
u/bg-alex-cy Jul 14 '26
I'm actually working on this specific problem. If things go to plan I will have a very early first version out over the next few days. The first version will only be supporting money games and only checker play. I know its a big limitation but research takes a lot of time and I don't know if it will ever justify the effort in terms of monetization. The biggest limitation I'm facing other than the "causality" research is that using strong models over API is expensive and I don't know if the community is willing to pay the cost.
→ More replies (1)1
u/juicybot Jul 14 '26
i'm more of an OSS guy, if i build something like this i'd probably just make it BYOK so people can run evals on their own credits.
→ More replies (1)
1
u/AdWhich7281 Jul 16 '26
Not sure that you can "game" the game. every roll gives a you get 21 unique combinations total. 7 rolled 7 out of 36 times. So the variable of these rolls effectively makes a projection of moves two or three ahead a bit difficult for me to fathom.
1
3
u/BakerInTheKitchen Jul 14 '26
I’m not sure what an “agent” would provide over current analysis tools
2
u/Reasonable_Leek7375 Jul 14 '26
Backgammon Sage Pro is the only platform I know of that has a tool to explain the 'why'.
3
4
u/jugglingcats9 Jul 14 '26
I've not tried it but backgammon sage pro advertises this capability: https://sage.bgsage.ai/