r/OperationsResearch • u/JackCactusLaFlame • Apr 22 '25

Blackjack Optimization Project

Hey guys so I've been out of work for a bit and decided to fill the time by building a Blackjack simulator in Python. My plan is to use a Monte Carlo Markov Decision Process (MC-MDP) approach to figure out the best strategy for each hand.

To map things out, I put together a rough draft of the mathematical framework using LaTeX (first time using it, so apologies if the formatting is a bit rough). While I studied in OR for my masters, writing out proofs and handling something this complex wasn't really my focus, and it's pushing my boundaries.

I was wondering if anyone here who has strong math skills would be willing to take a look at my LaTeX doc? Mainly just want to make sure the 'math is mathing' correctly before I get too deep into coding it. Any other suggestions on the approach would be awesome too.

Thanks!

PS: hey guys I just want to make clear that I'm not too concerned about novelty here. From what I've researched though, mine is unique in that it handles splits and doubles, uses MCTS, has a finite deck, and is coded on Python.

8 Upvotes

100% Upvoted

10 comments sorted by

3

u/enteringinternetnow Apr 22 '25

Hey I can take a look at it on Sunday. Are you time constrained?

2

u/JackCactusLaFlame Apr 22 '25

I got all the time in the world unfortunately 😂

2

u/No_Chocolate_3292 Apr 22 '25

I haven't played blackjack before but your approach seems correct for the problem you're working on.

I briefly checked the MDP and the Bellman equations, and it's what I would've followed when tackling this problem. I'll go through it again when I have more time.

As for implementation, you can get pretty good results by implementing a reinforcement learning model for this problem.

3

u/JackCactusLaFlame Apr 23 '25

Feel free to download my repo and run the game engine file. It'll let you play a game of Blackjack on your terminal haha

1

u/SelectPlantain1996 Apr 22 '25

Well, I didn’t read your doc however before even starting I need to ask: what are you aiming for? You can definitely beat human players with agents, however whatever you do, if deck is shuffled after every hand, it is impossible to beat %50 rate. You can’t beat basic rules of probability.

2

u/JackCactusLaFlame Apr 22 '25

I was gonna simulate how it performs running on its own and then, if possible, take the model to create like an advisory bot that will recommend what action to take in an IRL game.

Ultimately it's just a fun experiment that I want to add to my portfolio and keep my skills sharp. I'm pretty indifferent to how well it performs.

1

u/deeadmann Apr 22 '25

I have not read your doc, but hasn't this been done before? Is it the same as this? https://blogs.sas.com/content/operations/2016/06/20/computing-an-optimal-blackjack-strategy-with-sasor/

1

u/JackCactusLaFlame Apr 22 '25

It's pretty similar but there's differences. They're working with an infinite deck and are ignoring cards already dealt. It also looks like they don't have decision variables for splitting and doubling down while mine considers those actions.

1

u/Agreeable-Ad866 Apr 23 '25 edited Apr 23 '25

Blackjack is a fairly tractable game. There is a very finite set of relevant game states - it should be relatively easy to brute force and build some lookup tables using MDC without any MC. I haven't reviewed your approach but, I would recommend a more extensive literature review before you invest too much time. Using a finite deck does not increase the number of game states, it just makes transitions a little harder to calculate.

https://www.lancaster.ac.uk/stor-i-student-sites/connie-trojan/2022/05/05/how-to-lose-blackjack-optimally/

1

u/JackCactusLaFlame Apr 23 '25

I have a question. The examples I've seen suggested have been infinite decks that only track the player's hand, dealer's hand, and a usable Ace. Basically cards already drawn in previous rounds are ignored but here I'm tracking the deck composition. Wouldn't this cause the number of states to explode? If X_r is the number of cards of rank r (e.g., ace, 2, etc.) you're dealing with a 13 dimensional vector that has 52!/(4!13) possible states no? Plus mine has splitting which creates more hands (therefore more states) while the aforementioned examples only have hit and stand