DeepMind’s AI Trained For 5 Years... But Why?
Ғылым және технология
❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com/papers
📝 The paper "From Motor Control to Team Play in Simulated Humanoid Football" is available here:
www.science.org/doi/abs/10.11...
arxiv.org/abs/2105.12196
My latest paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Edward Unthank, Eric Martel, Geronimo Moralez, Gordon Child, Jace O'Brien, Jack Lukic, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Matthew Valle, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Twitter: / twominutepapers
Web: cg.tuwien.ac.at/~zsolnai/
Пікірлер: 860
A few years ago you helped ignite my passion of machine learning. Since then I’ve learned to code, have educated myself vastly in machine learning. I’m currently building my own small NN. Thanks for opening my eyes to something amazing
@TwoMinutePapers
Жыл бұрын
That is absolutely amazing, so happy to hear your story! Hope the neural network will go on to do great things!
@HiddenExp
Жыл бұрын
@@TwoMinutePapers You are big my fellow scholar
@bubbleboy821
Жыл бұрын
The first time I learned about them was SethBling'd Mario NN
@Litwinel
Жыл бұрын
@@TwoMinutePapers this must be a great feeling to know you are an inspiration for others
@electron6825
Жыл бұрын
@@ThatGuy-kz3fx neural nutwork
It would be cool if there were simulated sport competitions where teams used their own AI model against each other
@raylopez99
Жыл бұрын
They do this already with chess, computer chess competitions, the latest winner is AlphaZero-based Leela, just beat the reigning champ StockFish.
@SpaghettiToaster
Жыл бұрын
@@raylopez99 Didn't Stockfish win?
@TwoMinutePapers
Жыл бұрын
Loving the idea.
@cosmos3576
Жыл бұрын
I would like to see this also in FIFA.
@sebastianjost
Жыл бұрын
There are competitions like that. Not sure if there are any for sports, but I've definitely heard of some other competitions: - Minecraft village/ city generation AIs - Minecraft resource gathering AIs (get diamonds etc.) - Ticket to ride AIs (board game) - bad piggies/ angry birds AIs (not sure which one) - chess AIs ...this is what I remember without actively looking for them. If you look for competitions like that, I'm sure you'll find some.
To get a feeling of how difficult this is: imagine QWOP with 56 keys instead of 4, and you don't just have to run but to play football, in 3D instead of 2.
@dzambi
Жыл бұрын
@@eetm but your 🧠 is just a processor playing 3d qwop in the dark.
@InceyWincey
Жыл бұрын
To get a feeling of how difficult this is, try to consciously control all of the muscles that are required to walk. Or just appreciate the fact that your brain can do that for you while you think about more important things.
@sfurules
Жыл бұрын
@@dzambi I didn't expect this level of existential crisis on a chill Sunday....
@jamqdlaty
Жыл бұрын
Also the buttons are pressure sensitive. ;)
@spizzleyo
Жыл бұрын
All things should be related to QWOP
they should do this with real rules (throw ins and cornershots etc), and with 11v11, run it for weeks on different computers, see if they come up with some kind of great strategy, see what formation and stuff they pick.
@letMeSayThatInIrish
Жыл бұрын
And add stamina so they have to learn to economize their energy.
@softwarelivre2389
Жыл бұрын
And goalkeepers
@alihms
Жыл бұрын
And a referee.. who always seems to be one sided too for the real effect.
@user-eg7cz1oq9m
Жыл бұрын
@@letMeSayThatInIrish yeah... genetic, height, weight, heart illness, traumas :D
@nobodyishere
Жыл бұрын
@@alihms lmao
I notice that they still seem to move really unnaturally -- their upper bodies seem very flail-y. I wonder if that would get ironed out if they were given some cost to excess movement (just like humans get tired).
@Saotik
Жыл бұрын
I was just typing something similar before I saw your post. I even used the word "flaily". Adding fatigue to the simulation feels like it might have significant impact.
@budnick1
Жыл бұрын
Along similar lines, if there was some cost to getting hit, like being slow to move for some time, I wonder if there would be an emergent consensus to avoid causing damage to your opponent, like an emergent moral code. Would they even develop a tit-for-tat rule enforcement?
@nemonomen3340
Жыл бұрын
I was also about to say this. It seems to be a commonly overlooked issue when training AIs in movement.
@maelstrom2313
Жыл бұрын
They may be moving more efficiently than real football players. People don't always take the most efficient paths when moving, we have to consider extraneous social variables of how our movement looks. It's possible that it takes more energy to restrict the movement of limbs than to incorporate their inertia into the body's trajectory.
@miladragon
Жыл бұрын
@@maelstrom2313 if that were the case, real soccer players would move like this. No one cares about looking stupid if they win (cf. the Fosbury flop, which looks stupid but is standard, because it lets you jump higher). There's also no reason to think that these should be moving efficiently, because there's no incentive for them to be doing so (they don't get tired).
It's crazy how the AI just iterates and comes up with through passes, lobs and cruyff turns.
@jakab174
Жыл бұрын
i am a little bit sceptical, with this, there are probably thousands of hours of footage and they show the best most human like stuff, these are things that just comes from the noise. If the ai would be any good they would shoot the ball directly to the empty net every time, then it would learn its better to put one player to the goal
@yousuck6222
Жыл бұрын
@@DailyCorvid You dont need a ref with robots. That's the point.
I think the pre-training behaviour pretty much captures the behaviour of actual football players
@vasekbrezina2801
Жыл бұрын
Exactly :-)
@brooksgunn5235
Жыл бұрын
I was gonna say 😂
@theinacircleoftheancientpu492
Жыл бұрын
Ah, the bugs kick in sometimes.
@vixantenna
Жыл бұрын
From the expert opinion of someone who's never watched football before
@JH-ji6cj
Жыл бұрын
@Vixan knowing football, but not acknowledging flopping from a foul? Who doesn't watch football?: you
Imagine training an AI to do a task for 100 years in 10 minutes, then exporting it and importing it into a robot to achieve the task perfectly in the real world What a time to be alive!
@spinninglink
Жыл бұрын
Think i saw something about them doing that with a ping pong machine, right?
@CircuitrinosOfficial
Жыл бұрын
I wonder if they inject noise into the simulation to simulate the imperfections of reality.
@derekofbaltimore
Жыл бұрын
Your point holds but i dont think that doing things in simulated world corresponds 1 to 1 with the real world. It would still need lots of training in real world (because the simulated world wouldn't hold all the variables that the real world holds and those small inconsistencies add up)
@josesandv
Жыл бұрын
I have always ponder that if our universe is simulated, maybe it is in a supercomputer that does million of iterations, where every iteration takes milliseconds but for us is eternity
@el2746
Жыл бұрын
@@josesandv More like every millisecond does millions of iterations
Honestly the first one where all the players are writhing on the ground looks pretty accurate to me.
@KangJangkrik
Жыл бұрын
Seizure?
@KuZiMeiChuan
Жыл бұрын
@@KangJangkrik The other team hit me.
@dranirbanpal
Жыл бұрын
Yes, that's advanced training on what to do after a foul
@vibovitold
Жыл бұрын
neymaring the shit out of the game
@Soupie62
Жыл бұрын
If that was truly all you need do to play, I might stand a chance of qualifying.
*Imagine a Zombie movie/show where the zombies first start out writhing on the ground, and then quickly they learn to get up and walk, and then run, etc, etc*
@vibovitold
Жыл бұрын
and once the movie credits start rolling, there's Pink Floyd's classic playng in the background "...Hey, teacher! leave them kids alone..."
@thecousindeci1103
Жыл бұрын
bruh thats a really cool idea for a game. The longer you live the smarter the zombies get with ai, honestly it would be scary as shit when they all learn to run and look for you in houses. Something like project zomboid but at some point the zombies learn to coordinate.
I see some serious meme potential in this
@FenrizNNN
Жыл бұрын
We can make a religion out of this
@MarcillaSmith
Жыл бұрын
What is the game the AI is playing? It looks like non Americans trying to invent their own version of football 🏈
@LinggarMaretvaCendani
Жыл бұрын
@@MarcillaSmith Why do you call it football when you bring the ball using hands? Shouldn't it called Handball?
@speedfastman
Жыл бұрын
@@LinggarMaretvaCendani No it should be called soccer cuz it socs.
@MarcillaSmith
Жыл бұрын
@@LinggarMaretvaCendani handball is when you can't afford a racquetball racquet.
Even after all these years, you still amaze me with how understandable you make these papers for people like me. Thanks for really spending time in digesting this information to someone who's not in the field or can't allocate enough time to dive deep.
@TwoMinutePapers
Жыл бұрын
You are too kind - thank you so much! 🙏
This actually could be a really cool esports team. Like imagine if madden had AI agents to play the team. It was actually pretty entertaining to watch.
@michaelatorn8380
Жыл бұрын
I see great potential in deadly sport types played by AI.
@zapdart1803
Жыл бұрын
Worth checking out altered state machine and their upcoming games (FIFA AI League and AIFA)
@Ulexcool
Жыл бұрын
Nah
@MrUbister
Жыл бұрын
@@Ulexcool Hater
@brendanrodgers9753
Жыл бұрын
@@michaelatorn8380 any examples of what sports specifically, can't think of any other than extreme diving
It would be cool to watch the 50 days AI vs the 3 days just to really show the improvement.
I would absolutely watch videos of a bunch of ai players fumble around like this for hours.
@devfromthefuture506
Жыл бұрын
Me too and I would like to ask where I can find more videos of these
@zapdart1803
Жыл бұрын
FIFA AI League is releasing in less than a month. You will get your wish
@aaronconnolly1496
Жыл бұрын
Just go outside man
I want more Ai Football!!! I wanna watch a full match! Looks so fun. Imagine what they might do years from now??
8:22 **gets lightly tripped over, falls down and has pain seizures** The most realisting thing in the entire video.
Two Minute Papers, thank you so much for providing us such incredible content.
realistic human movement that follows the laws of gravity and moves on its own has always been fascinating to me
How epic is it gonna be when a fellow scholar makes a soccer game like FIFA but using these AI's
@zapdart1803
Жыл бұрын
FIFA AI League game is about to be released on the App Store
Boston Dynamics vs Real Madrid when? Also I want to see 100 vs 100 players and if this can be transferred onto soldiers like in Totally Accurate Battle Simulator.
I would love to find out with a model like this what the ideal play style would be according to game theory and physical limits of the players, but still having to follow all the rules. For example: Would it actually be better even for the goal keeper to move out and play in the field, vs. staying in the goal? I guess humans didn't figure out optimal gameplay yet for soccer and this could lead to new and crazy strategies.
@SweBeach2023
Жыл бұрын
It was my initial thought as well. Using AI we could try so many new strategies. And it wouldn't be hard to give the different AI players different properties to reflect the properties of the real players of a professional team. On eis a faster runner, another have better stamina, a third one have great control of the ball etc.
Just imagine the labour curve going down suddenly after a couple of weeks… nobody knowing why… and AI looks like it would start to discuss and talk to each other on the field rather than playing. THAT would give me goosebumps. ;D What a time to be here.
I need this to become a thing, like using real football matches to train the AI and have simulated matches between real teams based on how they play. I would love that.
Thank you especially for comments on those graphs. Big respect!
I wonder if it's possible to train certain skills first, like getting up, sprinting and kicking separately, then continue with the game training.
My heart desires a whole tournament with these little AI's. Crazy ragdoll physics and wild flailing bodyparts included. They are so fun to watch! Can you upload more footage?
I, for one, welcome our new AI football overlords.
these little foot ball player are so fun to watch i could watch that for the whole day
on multi-agent ai, do we have to relearn a strategy for 11 v 11 instead of the current 2 v 2? i assume so because in theory its a different game?
This is the kind of niche thing that I personally really enjoy reading in my own time, but it's so much better when you have a PhD holder narrating for you and showing you all of the nuance that you might have missed.
It would be awesome if they can also simulate some form of fatigue, some movements look too energetic or wasteful.
Physics agents are so interesting, I wonder if there is any demo of this that I can find so I can run it myself. Would love to train goalkeepers and play 6v6 or even branch out to something like fencing!
it would be so sick to see full teams progress and having a goalie learning how to goalie and adding in rules like offsides, out, fouls, penalties. would like to see what formations they would come up with or if they would stick with the all forward all back game plan
2:11 "Holy mother of papers!!" hahahaha love it. Great videos and papers 🤖
If this simulation gets more accurate, it could influence how football is played in real life. Imagine when the richest clubs can afford to create such simulations and optimise their players.
@bobisatelier
Жыл бұрын
they wouldn't need to be rich probably
I hope you have seen "Lawyer Explains Stable Diffusion Lawsuit (Major Implications!)" by corridor crew uploaded a few hours ago. He even uses your catchphrase at the end! What a time to be alive!
I'm fascinated by the game AI Roguelite and stuff like this video, so I'm subbing now - Look forward to seeing more of your content!
Wow, this video showcases the impressive capabilities of DeepMind's AI technology. The ability for the AI to learn and adapt to the complex rules and strategies of football in a simulated environment is truly mind-blowing. I can't wait to see how this technology will be applied in the real world and the impact it will have on the future of sports and beyond.
I like your storytelling, sir. What a time to be alive!
3:05 so cool how they visualize field control
Wow! The first steps of a Simulation World just started, can’t wait to see virtual peoples doing tasks like our own.
The way this dude pauses to extend EVERY diphthong. A-and. So-O. No-O. YE-es.
One of my favorite 'agent' training examples. So wild.
Absolutely fascinating. Thank you
I can't wait to see the Olympic AI Opening ceremony!
Could be interesting to set the standard conditions/physics for the match and then have two teams train their NN best possible, aftwards pitting the AI teams against each other. Wonder if there could be upsets like irl or the better team would always win
2:20 We used to do the same trick when we were kids, passing over the wall when we played soccer.
Remarkable. Ex footy player here, wondering how long till these analyses show how Messi's additional value was in the distribution of labour - a 'team player'.
I wish there was like a place I could go and watch these AI do their thing in real-time, I could watch that stuff for hours.
The fact that it learned to embellish an injury, only to pop right back up is great
Ayo very cool video! But why does the "Division of Labour" Chart start so high? They didn't Teamplay in the beginning, did they?
The talking sound so robot like. Its like you have all the words with your voice already recorded and then you just type the words which automatically creates the voice
I think one of the major improvements in the last few years we've seen in AI training (besides training size/time) is how we train intermediate goals to speed up training. If we simply took the untrained AI with the joint body, and asked them to learn soccer, it would take forever with minimal improvement. But as we've seen in this video, they first train the AI to enable it to walk, dribble, and kick the soccer ball. Once the AI knows how to perform general ball handling tasks, it can use that understanding to more quickly learn to score goals. This might just be my perception though.
@vibovitold
Жыл бұрын
the next logical step would be (and it's probably already done to some extent, in one way or another) to split the AI into a "teacher" (supervisory) and a "pupil" (learning) AI, whereby the former one would be in charge of figuring out and setting the incremental goals for the latter to accomplish
i love your content bruh, long time fan! ölelés!
How do you train AI like this with UE or Unity? I dont understand how that works and I coulnd't find any tutorials on it but Im not sure what to look for..
They're using their body to shield the ball and then turning around to beat the pressure from opponent... Wow.. That's something one of the best midfielders of all time Xavi did often too... Incredible
The pure PANIC in their running HAHAHAHAHA
@enderyu
Жыл бұрын
Don't forget the guy having a mental breakdown at 4:22 and then immediately snapping back to the game
@axelwickm
Жыл бұрын
@@enderyu It is all very relatable.
One day there's gonna be a better way to 'start' these training projects. It just seems wrong that they should start with such little knowledge. I'm sure something will arise as we work towards the future. By the way this was damn impressive! Just the mere fact that it could side-step and through-ball shows how much of an understanding it really has about the game, a surprisingly deep one!
@bingchiIIing
Жыл бұрын
@@DailyCorvid loving this
Just imagine if this ai was put into a robot and you had to play soccer against it. It'd be horrifying with how they move - imagine if they even looked like people 💀 Still absolutely insane progress with this kind of physics-based ai agents! It's crazy!
@asrar4907
Жыл бұрын
you do remember the part where he said that the ai trained without a refree? Gosh you need an ambulance
@davidwuhrer6704
Жыл бұрын
RoboCup Humanoid League
@MACHINEBUILDER
Жыл бұрын
@@asrar4907 Ahha yeah you'd definitely need at least an ambulance after playing against these
@dagdbot83
9 ай бұрын
I think they did that already
HOLY MOTHER of Papers,... Hahaha Love u Man.
0:53 me when mom says I can't have another pack of fruit gushers until I eat some broccoli
The through ball at 4:29 is amazing 🔥🔥
I would have loved to see at least a month or 2 into this. Bet I would learn something great from that
it would be interesting if they added some stamina function, so they run more realistically and not swings their limbs all over the place.
@ChristopherCricketWallace
Жыл бұрын
yeah. I would like to see the rules built-in, as well. Let the AI learn in the context of the real game with out of bounds, and penalties, corner kicks, etc.
@drawmaster77
Жыл бұрын
@@ChristopherCricketWallace I mean this maybe overdoing it with the rules, but at least have them run more realistically not like they are spazzing out with limbs flailing all over the place lol. Maybe add some cost to limb movement which factors into their stamina bar and when stamina is low they run really slow which would affect their performance and that way AI would learn to be more "efficient". Just throwing some thoughts out there.
1:13 This is proof that not knowing how to play football can cause extreme cases of seizures.
I'm quite curious why many of the graphs have staggers going down, instead of continuously improving. Any idea what that might be about? For example, in "Passing Range" it goes down inthe 1 week range at 6:35
@coder0xff
Жыл бұрын
It's complicated because you're working with many variables. Imagine you're going to your friends house but you don't have a map of the roads, just a compass that points to your friends house. You go down a road that gets you closer, but it's a dead end. You have to go back the way you came a little to find the better road that will get you even closer. If we imagine that hip angle is the x axis of a map, and ankle angle is the y axis, then we might settle into an effective ankle angle. But then hey, if we tweak the hip angle a bit we can get even better, but the ankle angle needs to be adjusted again. It's like this, but with way more variables.
@ariseyhun2085
Жыл бұрын
@@coder0xff thank you so much for this explanation!
Would be amazing to see this as a combat sport or battle simulation.
They're training them on rotoscoped humans playing, so was the feints with the footwork introducing "noise" that was creating the initial shaking/quivering? When they're running they take lots of tiny steps, which makes me think that's a latent trait of the human data set foot-faking distorting the training?
6:40 its interesting how that curve looks similar to the Dunning Kruger Effect
testing 3v3 ir 5v5 matches would be very interesting as well
AI scientists should have a competition to see whose model can train their players better and have an actual virtual football cup.
Happy noises from Switzerland!
You voice are a IA too? o.O
Pure gold, they play like half of my classmates on PE :D
how powerful of a processing unit did they train for those 5 days of drill training?
I studied multiagents theory at university and I can't wait to see this expermient go with more players ! BTW if anyone has sources about theory of agents interactions in tthis context
The red teammate at the top looks like something out of a nightmare 5:55
It seems as if introducing some sort of stamina limit will fix a lot of those weird movements. And of course make more stressful body positions use more stamina etc
I always wanted a physics-based walking system in a videogame and although I never thought I'll be here for it, this makes me think I just might. Counter-Strike 2 where you have to be careful about how fast you're running down the stairs or walking on mud? Yes, please.
The beginning reminds me of Italian soccer players when they play out their drama scenes
1:00 the footballers like babies he crying
4:22 that's pretty much Messi's signature move.
I have always enjoyed your videos and find them way above my head. But I am slowly learning ML. I still can't link what I watch here to what I can do to get there. ... I am looking for projects to work ob
That red AI trying to feint injury crack me up
Ok, now put the code into 22 boston dynamics robots and let's go
0:56: "How much beer do you need" Him: "Yes"
Imagine if they could do AI training for robot control (in real life)...
4:22 The blue guy having a quick panic attack off to the side before returning to the game
Love the videos!!!
Very interesting If they could also simulat the muscle exhaustion or energy usage for the movement, they could have a more efficient way of running. Because for now they run very crazy, it seems they have these kinds of mouvements from the infinite energy they have to move in the simulation. Maybe also some injury system. And of course, like mentioned in the video, a referee.
1:05 That poor guy in the middle of the field trying so hard to get back up!
imagine they use the fifa offside detection ai thing to train this ai further
And to think how jaw dropping it was to see open AI dominating a 1v1 against pros in Dota 2 years ago. Unreal how far it’s come and it’s only going to grow exponentially from here
The way they move and play looks so funny and too much training will probably iron out that character.
Hello sir ! I want to know how good is general purpose AI against specialist AI ? Is general purpose AI is like "Jack of all trade and master of none " against specialist A.I ?
@coder0xff
Жыл бұрын
There aren't currently any general purpose AIs, but it seems reasonable that a specialist AI would be able to do certain things better than AI general AI. It's not really a "specialist" otherwise.
this channel is so underrated. Also its quite scary
Those pre training moves are what I look like when I play sports
I hope they will continue the training. I want to see after 100 days and 1y
I love the pretrained red guy in the middle of the pitch. Give him to glow sticks and he looks like he's at a rave.