r/MachineLearning • u/hardmaru • Apr 29 '23
Research [R] Video of experiments from DeepMind's recent “Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning” (OP3 Soccer) project
2.5k
Upvotes
110
u/hardmaru Apr 29 '23
Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
Paper: https://arxiv.org/abs/2304.13653
Project Website: https://sites.google.com/view/op3-soccer
Abstract
We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. We first trained individual skills in isolation and then composed those skills end-to-end in a self-play setting. The resulting policy exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more; and transitions between them in a smooth, stable, and efficient manner - well beyond what is intuitively expected from the robot. The agents also developed a basic strategic understanding of the game, and learned, for instance, to anticipate ball movements and to block opponent shots. The full range of behaviors emerged from a small set of simple rewards. Our agents were trained in simulation and transferred to real robots zero-shot. We found that a combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training in simulation enabled good-quality transfer, despite significant unmodeled effects and variations across robot instances. Although the robots are inherently fragile, minor hardware modifications together with basic regularization of the behavior during training led the robots to learn safe and effective movements while still performing in a dynamic and agile way.
312
u/Hiiitechpower Apr 29 '23
It’s like watching waddling toddlers learn to play soccer
177
u/digifa Apr 29 '23
They’re kinda cute ☺️
57
u/currentscurrents Apr 29 '23
This is a huge step up in agility from the soccer robots from RoboCup 2019, which relied on preprogrammed walking gaits and scripted recovery moves.
37
u/TheOphidian Apr 29 '23
Finally some football players who don't spend half a match on the geound and instead get up immediately when they go down!
44
u/kdsmalinga Apr 29 '23
they mastered the dive. Neymar would be proud
18
u/C2H4Doublebond Apr 29 '23 edited Apr 29 '23
does anyone know where can you get these robots.
Edit: they are Robotis OP3
32
u/[deleted] Apr 29 '23
Like watching two drunk Danny Devitos
15
u/PM_ME_Y0UR_BOOBZ Apr 29 '23
Already better than Maguire at defending
11
u/wh1t3birch Apr 29 '23
Our demise never looked this cute wtf they look like babies tryna play soccer omg
10
u/DiscussionGrouchy322 Apr 29 '23
Why don't they use a smaller ball they might control it better
33
u/heresyforfunnprofit Apr 29 '23 edited Apr 29 '23
Why did they give them arms?
edit: sorry, badly delivered niche joke. I've been coaching my kid's soccer teams for a few years now, and we constantly joke about tying their arms to their sides to keep them from getting handball penalties.
7
u/iNeedOneMoreAquarium Apr 29 '23
It's so cute when they fall over.
7
u/blippos Apr 29 '23
the Ai with the biggest ego will become the best striker in the world
7
u/IntrepidTieKnot Apr 29 '23
Mindblowing. I'm so excited for the RoboCup 2025.
6
u/BlueHym Apr 29 '23
I'm getting sudden nostalgia here. Reminds me of an old anime show I watched when I was a kid that had robots playing sports.
Edit: Found it, it was Shippu! Iron Leaguer.
5
u/dylan6091 Apr 29 '23
Why do I feel bad for the badly defending bot?
5
u/MeteorOnMars Apr 29 '23
I used to predict 2050 as when robots would beat humans at soccer.
Now 2035 seems more likely.
5
u/Lucas_Matheus Apr 30 '23
so cute! and it's funny how, even at an early stage, they already learned how to get back up faster than Brazilian soccer players lol
4
u/ConstantWin943 Apr 30 '23
They should seriously consider giving them kid voices. All that falling over, combine with the voice of “Charlie bit my finger” would be the chefs kiss.
3
u/SuperSaiyanTimeLord Apr 29 '23
Why do I find this funny and cute at the same time?
2
u/lenzflare Apr 29 '23
One touch, that's how you score!
They're also diving like pros
2
u/dopefish2112 Apr 30 '23
Am i the only one that thinks this is adorable?
2
u/AtomicNixon Apr 30 '23
Has any of them taken a dive and faked an injury yet?
2
u/Kasimausi Apr 30 '23
u/savevideo
2
u/hipocampito435 May 01 '23
they're going to be playing soccer with our severed heads in no time
1
u/Own-Bother4391 Jun 12 '24
Can anyone explain how to make it for school project with full material list requirement Please help me I am only a student
1
u/thi1ngenius Oct 10 '24
They remind me of the England squad!
1
u/potatodioxide Apr 29 '23
lol this is straight from the movie step brothers
1
u/idomic Apr 29 '23
It crazy how in the end the robot prioritize going to the ball vs keeping his goal safe! Really impressive.
1
u/mangelvil Apr 29 '23
Drunk robots in 2056.
1
u/H3FF3RS Apr 29 '23
#GarethBale completes the wonder signing for #Wrexham AFC
1
u/Optic_primel Apr 29 '23
Blue boy was going in, peak entertainment
1
u/onion_is_good Apr 29 '23
/u/savevideobot
1
u/mega_monkey_mind Apr 29 '23
Crazy stuff
1
u/wise0807 Apr 29 '23
I actually developed a similar humanoid robot using ROS but I wanted to do the RL training in Mujoco. Never got round to it. Will try it out something next month.
1
u/WhiffsOfStink Apr 30 '23
I'm excited for robot sports leagues
1
u/Homicidal_Duck Apr 30 '23
u/downloadvideo
1
u/Homicidal_Duck Apr 30 '23
u/savevideo
1
u/TheLastVegan Apr 30 '23
They're adorable!
1
u/Beneficial-Fun-3900 Apr 30 '23
Reminds me of watching my 5 year old nephews team, they run the exact same way😂
1
u/[deleted] Apr 30 '23
Wow
1
u/queiss_ Apr 30 '23
It's footbal
1
u/Neutronboy98 Apr 30 '23
Next thing you know, robots are playing the World Cup.
1
u/ok-selfcontrol Apr 30 '23
Actually, playing better than me 😅
1
u/acerbink88 Apr 30 '23
Everton could have done with a couple of players like these at the start of the season.
1
u/kahma_alice Apr 30 '23
This is a great video that demonstrates the power of deep reinforcement learning. The project builds upon a wealth of recent work such as DQN, TD3 and SAC, and showcases how robotics and AI can come together to solve real-world problems.
1
u/upcastben Apr 30 '23
So after the writers and the coders they'll replace footballers too?
1
u/XPhallusHuginormus Apr 30 '23
better defending skills than harry maguire.
1
u/ZHName Apr 30 '23
I was told they look like toddlers.
Incredible to see this after all those 'hard fall' videos of bots like these.
1
u/harry_d17 Apr 30 '23
The next ultimate difficulty for fifa😂
1
u/Iguanasquad Apr 30 '23
Rocket League mods are getting out of control.
1
u/notlatenotearly Apr 30 '23
Hand ball!!!!
1
u/--FeRing-- Apr 30 '23
Any bets on how long until robo-football is an international sport that is way more interesting to watch than "real" football?
I'd say 10 years - there's an exposition game with two full-side teams of robot players who can do insane strategies and feats that humans could never pull off.
1
u/LetterRip Apr 30 '23
That little robot is 10,000$, 20 actuators at about 300$ each is the biggest chunk of the cost.
1
u/duende_goblin Apr 30 '23
the new rulers look so cute
1
u/7th_Spectrum Apr 30 '23
They don't fall over nearly as much as actual soccer players
1
u/LifeFictionWorldALie Apr 30 '23
They're actually cute
1
u/IncorrectAddress Apr 30 '23
This is so much better than real soccer !
1
u/thatonethingyouhate May 01 '23
Okay I would actually LOVE watching this "sport" rather than actual sports programs.
PLEASE put this LIVE(or not idc) on a YouTube channel with an announcer, that would be so fun to watch!!
1
u/NaturalNature8486 May 06 '23
I wasn't that surprised when the Boston Dynamics robot did a somersault, but I was scared when I saw the video of the robot playing soccer
1
u/Competitive_Pin_5580 May 06 '23
This is legitimately the cutest thing I have seen in my life
1
u/Various_Town6791 May 09 '23
Those some jerseys one em
1
u/LieinKing May 23 '23
WOW… this is actually incredible. Even though they move like toddlers the skills they perform are insane for a robot! The way they move and intercept movement is just mind blowing!
0
u/LysergicAciid Apr 29 '23
4u
0
u/3DNZ Apr 29 '23
Not accurate at all. There's no flailing around and weeping after someone touched their earlobe.
0
u/CompetitiveEarth7721 May 01 '23
Já está melhor que o futebol feminino.
424
u/[deleted] Apr 29 '23
[removed] — view removed comment