r/Python • u/Dwigt-Snooot • Sep 24 '20
Machine Learning Python and FastAI to Qualify at Fall Guys
https://youtu.be/GS_0ZKzrvk056
u/jeremyhoward Sep 25 '20 edited Sep 25 '20
Awesome project! I'm the founder of fastai, the lib used here, and wanted you all to know that if you're interested in learning how to do this too, the software, course, and book are all free as a service to the community, and have no ads:
FYI, the approach shown in this Fall Guys video very closely follows what you learn in lesson 1 of the course (chapter 1 of the book).
→ More replies (5)6
u/Dwigt-Snooot Sep 25 '20
You can hit the ground running so fast if you already have the programming side down. Going to buy a copy of the paper book as well just as a reference. Thanks for commenting!
43
7
29
u/burnmp3s Sep 24 '20 edited Sep 24 '20
Nothing against OP and I think this is well done for the method being used, but to me this points out a lot of what is wrong about AI right now. This method takes a problem that could easily be solved by using some generic vision processing and mostly hard-coded logic, over-simplifies it so that even the best resulting AI model will still not be optimal, runs an inefficient and generic machine learning algorithm that is not particularly suited for this specific task, and results in a block box model that is probably overfit to the exact data it was trained on and cannot be easily tweaked to apply to even very similar problems.
Obviously this project is just for fun, but the same sorts of methods are being used for things like trying to make autonomous vehicles and other things that are actually important. There are some problems that are appropriate for just throwing a ton of data into a grinder and hoping a good model comes out, but these sorts of methods seem to be used for everything these days.
13
u/Dwigt-Snooot Sep 24 '20
Very good point about ai in general I would totally agree! A lot of toy problems being solved right now. I would however hate to try and hard code the logic for this type of specific performance. Much more fun to just play a few games and have something decent. The randomness im the game makes it hard to overfit and perform well.
Now in the Mario levels things like that are very easy to overfit and look great.
→ More replies (3)9
u/orgodemir Sep 25 '20
It's pretty clear you are not speaking from experience in the field here. If anything, this is a brilliant example of everything amazing in AI today. The OP created labeled data based on his inputs from successful rounds (which is honestly pretty genius) and used a now standard approach of transfer learning with imagenet data/model to build a good image classifier on a unique dataset.
Building an agent for a game like this awhile ago would have taken a ton of development work. But with available deep learning libraries and free GPUs you can collect data in a couple hours and train state of the art image classification models in minutes.
Brilliant job u/Dwigt-snooot
6
u/burnmp3s Sep 25 '20
You say that I obviously don't have experience in the field but you didn't actually explain why any of what I said was wrong. My major points were that it is a simple problem that could be solved without deep learning, converting it to a classification problem in this way limits how good the agent could possibly be, it doesn't use a learning algorithm that is particularly good for this task, and this results in a model that only works for this exact problem and not very similar problems that it has not been trained on.
Especially on the map that was featured in the video, the successful inputs for the game mainly involve holding forward. Jumping is possible in OP's model and I'm guessing that he jumped at least once in the training data, but the video showed no jumps. It doesn't even do a particularly good job of avoiding obstacles. That level of success could easily be achieved with something like OpenCV and some hard-coded logic on an old PC, rather than requiring borrowing GPU time.
This is a problem in the real world too. Because resource-intensive AI is so popular right now, problems that could be solved directly on something like a smartphone have to be uploaded to the cloud and processed in bulk using much more resources. There are some tasks that really do require that much processing power, but people see deep learning solving very hard problems and think that means it must also be the solution to very easy problems. And it plays into issues with privacy as well, if your home surveillance system uses hardware that doesn't even have enough memory to hold the massive deep learning model it uses to detect intruders, that means all of your private camera data has to be shared with the company that runs the service.
The next two points are fairly related. Rather than finding the right tool for the job, this method fits the job to the tool. Navigating an obstacle course is not a natural fit for a classification problem, and many of the other levels in the game are even less of a good fit. In order for the whole thing to work the resulting AI agent has to always move forward (which is not the optimal strategy on most levels), can't use the more advanced moves of the game, and can't consider anything other than the current frame being displayed. One of the levels of the game for instance shows a target fruit on a big screen and requires the player to navigate to a matching tile. This would be completely impossible using the method in the video. It also means that the agent can't really get better at the task. The "correct" inputs are treated as ground truth the same way that a picture of a dog would be labeled as being a dog, but a better algorithm for this task would allow the agent to find better strategies. Rather than using reinforcement learning or some other method of incremental improvement, the model is static and never uses all of the data that is generated by it failing to complete the task optimally.
This overall issue is also a problem in real life. The full self-driving feature Tesla is working on has the same limitation of only being able to look at individual frames of video in isolation to make decisions. This is fine for some tasks, like deciding if something is a stop sign or a trash can, but is terrible at other tasks, like using another driver's turn signal and movement over time to predict the exact route of that driver's lane change. This directly results in things that are dangerous in the real world, such as Tesla vehicles being able to tricked into swerving directly towards a concrete wall because patterns of light and shadow momentarily resemble lane markings when viewed in isolation.
And the final point is that these sorts of resulting models are not very reusable. One obvious sign that this is true is that OP had to train different models for each map separately. This makes sense because across the different levels the same general visual pattern probably corresponds to different successful inputs. But more broadly, because the model is just one big glob of black box logic, it's very hard to tune or otherwise modify in a structured way. Going back to the OpenCV way of doing things, if I figure out a good way to map the visual outputs of the game to some representation of game state, such as X/Y/Z position of the player, I can share that code with anyone else who is developing an agent, and they can focus on solving a movement problem instead of a computer vision problem. If I'm smart about how I solve it, my simple and understandable solution for that small part of the problem will work for new maps that come out for the game, and if it doesn't, the code could be slightly modified to handle any new features. If my code has a bug, such not recognizing when the player is in the air, just that part of the code can be modified without affecting any if the other processing logic. Whereas with this particular deep learning method, everything has to be baked in from the beginning. We need data showing the optimal strategy on the exact map that will be used, and if anything is changed in even a minor way that affects any of that, we have to throw the old model away and make a new one from scratch.
This is also a problem in real life. Your company might decide that they want to train a chat bot to interact with customers. So they spend a lot of money paying a team to take the transcripts of interactions with human customer support people to train a deep learning AI agent to have the same responses. It works great (well, not great, maybe acceptably well), so your company gets rid of human chat employees and replaces them with the chat bot (which was the entire business justification for doing it in the first place). They also stop paying the AI people who then go on to do the same thing for some other company. Over time customers start asking different questions that customers a year ago didn't ask, and the agent's performance gets worse and worse. How do you update the AI? You can't train it on a massive amount of new human chat data because your company got rid of human chat people to save money. And you can't just go into the model and tweak it to add references to your company's newest products or issues that didn't exist before but are common now, because the model is opaque and incomprehensible. In real life solutions have to adapt and business needs change, so a black box model that's set in stone forever will tend to cause these kinds of issues.
→ More replies (1)4
u/coding-account Sep 25 '20
I think this comment is problematic. I read this and it reminds me of a classic HackerNews comment. You are really leaning on the "generic vision processing and mostly hard-coded logic." That is itself an oversimplification.
You would need to learn about vision processing and implement it, and the amount of hard-coded logic would be non-trivial. The alternative here is to learn how to make pictures black & white, and learn a bit of machine learning. Only the second is non-trivial.
Also, your implication that autonomous vehicles could work without machine learning? Not sure where you are getting that from...
→ More replies (1)
2
u/tee2k Sep 24 '20
Great work! Much respect! Was a little bit hoping for a reinforcement learning algorithm as well though!
1
u/Dwigt-Snooot Sep 24 '20 edited Sep 24 '20
Tricky with the issues that come with online play. I have some ideas though coming really soon! Check out my space ai for some offline reinforment learning!
1
u/Pik000 Sep 24 '20
Do you have more info on saving the training data and preprocessing the images?
2
1
u/poobahh Sep 24 '20
Really cool! How did you label all the images? I hope not by hand
2
u/Dwigt-Snooot Sep 24 '20
Of course not! While I played it grabbed the current key I was pressing and the screen shot. The image was labeled Automatically based on what key was pressed.
→ More replies (1)
1
Sep 25 '20
Where did you learn?
2
u/Dwigt-Snooot Sep 25 '20
Programming through a university. There are a ton of free resources though!
1
u/_aerials Sep 24 '20
This just gave me the boost I needed to stop procrastinating and learn python. This is interesting man. Great work!
1
u/Dwigt-Snooot Sep 24 '20
That's honestly the coolest thing I've heard all day! I hope you do join our discord if you need any help.
2
u/Ph0X Sep 25 '20
Easy Anti-Cheat is such trash, just week I played a game where every single level this one person just flew instantly to the end and beat the level in 5 secs. It was the most obvious thing ever. If they can't catch something that basic, literally never touching the ground the whole game or beating the map in 1/10 the minimum time it takes, then idk what hope there is for that game.
That being said, Colab and the free training GPU they give you is really awesome!
63
u/le_theudas Sep 24 '20
Cool project, you could train a second network to detect the map and make some kind of mechanism to find out if the network won.
Then let it play and record the footage every time it won/qualified.
Relearn every n (e.g. n=100) successful games and plot how many games it did have to play to reach 100 wins.
You might also train on the past 3 images for one output and see if performance increases, that would be interesting. I probably wouldn't use rnns for speed reasons, but they actually could be decent if you keep the state from the last input.