WTF??? an AI drifts cleanly on hockolicious!??!??!

@kilvesx7924

8 ай бұрын

The most frightening thing about AI is that it's absolutely terrible in everything until it's not, and when it's not, it absolutely trashes every record ever. It's everything or nothing.

@bbbndddl

8 ай бұрын

It's not really frightening when you consider that it has no chance on levels which it didn't see before.

@kilvesx7924

8 ай бұрын

@@bbbndddl until it does. And when it does, it'll probably beat every human driver ever no matter the track

@linesight-rl

8 ай бұрын

Does it ? Have no chance ? What would be a good benchmark for tracks it has never seen before ? Let's say it is able to finish 90 percent of maps that it has never seen, is that any good ? Does it need to do bronze, silver, gold or author medal ? Curious to have your expectations, you know, for a future video 😁

@bbbndddl

8 ай бұрын

@linesight-rl what I meant was that it doesn't have much chance at performing as well as top players without any further training. This map is pretty uneventful. I'd say that AI having a top 100 time with a few hours of training would prove that Trackmania can be solved in the near future. Right now, I think it's highly overfitted to this particular track, and that's why it's so good.

Please try a trial map, even if only a small one. I would love to see what the AI can achieve on tricks that take hours for humans to perform. Maybe oachkatzlschwoaf on ice? Or only a segment?

@thomaslomba5181

8 ай бұрын

it'll take months but i kinda wanna see that lmao, or some easy kacky xdd

@kylehart8829

8 ай бұрын

I second this for sure, the reward function would be kinda hard to make and would likely prevent shortcuts and such but it would be extremely cool to see regardless.

@limitlesscord7319

8 ай бұрын

this

@mathmoril

8 ай бұрын

problem with kacky is that its black and wight often, you either get it or you dont, so setting good rewards for getting close is SUPER hard, if they could do that i'd be super impressed.@@thomaslomba5181

@uncleenk

8 ай бұрын

this is a great idea, though an rpg seems easier for it.

I think the next step should be meta-learning, so that agent can start learning on the new map without weights resetting. RL^2 is the simplest meta-rl algorithm, give it a try!

@peterranney9488

8 ай бұрын

Absolutely this. It might be really tough to integrate because it seems like a lot of the reward system that you use is based on learning the track, but if you can implement some form of meta learning it will cut down on your work per map by half at least. This would then free up more computing power to attempt higher frame rates while keeping the same amount of time to get results. Great work!

@gregzer

8 ай бұрын

Yes maybe I didn't understand, if they tried to make the AI trained on the previous map play this map it wouldn't work ? Would the AI just try to follow the path of the previous map ? I'm confused

@HyzerFlip

8 ай бұрын

​@@gregzerThe results from the inputs the AI is getting are tailored more specifically to this track. If you change the track, the AI would probably do "ok-ish" depending on what type of learning they used here, but it definitely wouldn't be optimal at all like we see in this video. Think of it loosely like learning to comprehend English, and then being told to read and comprehend Spanish. You've trained for so long to read English and comprehend that, but now you have a different problem you're trying to solve with (mostly) all the same types of inputs. Sure, you can "read" the Spanish, but you won't really know what to do with it to get any value out of it. It's not exactly the same type of thing, but the general idea is there.

@jjjoker5766

8 ай бұрын

@@gregzer The AI doesn't learn by doing random inputs until it finds ones that work. It instead looks a an image of the screen and tries to predict what it needs to do next, the longer it does this for, the better it gets at predicting

@nephandi2316

8 ай бұрын

​​​@@jjjoker5766 Sure but with a reward function running on 7 second, it's likely that the AI's strategy toward the end of the training is to find the closest picture from current one on the better previous runs and deviate very slightly on the 140 next inputs (20 fps x 7 second). Which means that if you give it a new map to learn, it will get confused cause it will try to replicate parts of the 1st map it got trained on, and the more maps you try to add the more confusing it will be, which means it will perform worse on the maps where it previously got close to WR while never reaching such times on the new maps. This is where meta-learning can provide good performance improvement, the IA would be less specialized and a bit more flexible, while (with a fair bit of luck) saving some data so they could improve fps or picture quality.

Well failing 53.xx on past streams I did not expect the AI to do so well to literally equal my time! Such a smart way of explaining and running this experiment too, mindblowing work!!!

Man you have to find a way to get yourself in the spotlight because this is amazing content and you deserve more than a couple 100 subs

@TheSuperappelflap

8 ай бұрын

Just send the video to wirtual on stream, i dont use twitch so im not gonna do it

@kevinquintana3085

8 ай бұрын

Let time do its job, like the old days, it's good content after all...

Thats awsome! Well done. But my bet for the best further improvment is not the framerate, but the resolution. With higher resolution the AI should be able to take the corners more tightly

@nanoder7te

8 ай бұрын

If performance is an issue with higher resolution, it would maby be an option to ignore parts of the frame? Like ignoring the top of the screen and what is located left and right of the car, to not bother about unnesesary pixels.

@Skycrafter_

8 ай бұрын

@@nanoder7te maybe even the shape of an eye is enough data

@kylehart8829

8 ай бұрын

Higher resolution is already gained in a way by higher frame rate. In the same way you can see through very small vertical slots by walking past them, the AI can use pixel data from one frame to the next to determine very precisely where the walls are and drive close to them. Increasing resolution would be more costly as far as performance and likely would show less results than framerate increases.

@vitorrodriguez4278

8 ай бұрын

frame rate is also very important, in the video the AI can only use a combination of 20 different key setups per second. With higher framerates, this number will increase so there is a lot more space for improvement

@chezeus1672

8 ай бұрын

@@vitorrodriguez4278 makes me think of riolu... that's basically what a higher framerate would mean for the AI.

This is incredible to watch. I also love how you intuitively explained how the learning system works and all, that's very cool

@linesight-rl

8 ай бұрын

Thank you very much!

Absolutely phenomenal ! I will eat my hat if an AI ever finishes Deep Fear respawnless (or at least a couple checkpoints) with that same learning method, prove me wrong !

@marczeslaw11

8 ай бұрын

Noted

@psychoedge

8 ай бұрын

Noted

@Takyodor2

8 ай бұрын

Note to self: return to this comment section with a tasty hat once AI finishes "Deep Fear" respawn-less.

@evgiz0r

8 ай бұрын

Should be easy for an AI if he can watch the records 🤣

@Mitin619

8 ай бұрын

AI wtf man just copy the wr..

this is amazing! i'm relieved to be part of the 23 who outclassed the AI x)

This is so cool! I always though ai driving in tm would never be this optimized. Well done! 🏁

It's brilliant! Well done, you're super good ! Good luck for the rest

I love how your videos related to AI with trackmania is one of the main reason why I'm passionnate about AI now that I know how silly I could be if I could hold this much power into making my own little cars go vroom. Thank you for this video, I loved it !

Very interesting to see the improvements. Well done and very well explained. Congratulations on this achievement!

It is important to note that this model wasn't trained to play Trackmania, but was trained to play this one map. It has most likely just remembered what each turn looks like and the optimal way to take it, rather than figuring out how to take each turn as it sees them. A good way to train a neural network for this sort of role is to give it lots of pro and intermediate level replays of different maps and have it learn to predict the player's inputs. Then after a while of training for this, you take this model and make it do its own runs on lots of different maps and reward it based on speed for further training beyond what humans have achieved. This is how AlphaStar and AlphaGo were trained. Alphastar is a Starcraft II AI that is the best in the world and AlphaGo is a Go (a board game more complicated than chess in strategy) AI that is also the best in the world. They were both made by DeepMind, one of the biggest AI research companies in the world, owned by Google. One last interesting bit of information, it was DeepMind who made AlphaFold, the AI that solved the protein folding problem and will likely win a Nobel Prize.

This is incredible, i love this. I am studying AI in uni and i hope, with more experience, to be able to make fun projects like this soon. I can't wait

I would love to see a more technical video about this AI, like what/how you got game information, which infomation you have used to feed the ai, and drawing GUI into the game window, awesome video btw!!!!

Not only a super impressive time by the AI, but also a sick video

This is a solid success. Well done mate!

That’s genuinely so amazingly cool that we can watch a computer program do things like this, and also things like TASes.

This is insane. Amazing work! I'm interested to see it learn to do other tricks on its own like wall bangs, air brakes, speed drifts, ramm hit, etc, or maybe even start finding cuts that no one sees.

very impressive indeed. I love Hockolicious and every insight is welcome

I love how it went from „Struggles to get over a bump“ to „Hard for a veteran player to achieve“ real quick

Amazing work 👏 🙌

Never expected AI to get this far this fast. Awesome job! Makes me wonder what is to come. Loved your explanation too

@mooncalf_4534

8 ай бұрын

The caveat is that it's making its improvements over people via being able to perfectly speedslide in a way a human physically can't. It's fundamentals are a lot weaker.

this is so cool, definite new sub here :O

Wow, very impressive progress!

This is incredibly impressive!

Kindheitstrauma would be incredible to see how the AI handles extremely high speeds, wall rides, and loops. Additionally a kacky map would be absolutely insane, but honestly, from a layman's pov, Idk if it'd even be possible 😅

@TheSuperappelflap

8 ай бұрын

Anything is possible, you would just need to define the reward function differently to take into account the unique approach kacky requires.

Wow! this is incredible!❤❤❤ I would love to see you teach the AI glitches, such as bugslides, or nosebugs. Maybe it could even learn to noseboost!

Keep up the good work!

this is super impressive!

Really good video !!

Well done Linesight, 400 subs overnight!

thats amazing work i cant believe it achieved those times. next up map suggestions ofc kacky

this channel will explode if you keep doing ai racing games

I love that one AI training's most primordial arenas is its use in driving. Automating travel is, ironically, one of those things that splits people down the middle in terms of modern issues. Some people see driving as something that is dangerous to automate because of potential injury or fatality, others would rather sleep on their hour-long commute to work if they're far from home. Then, on the opposite axis of the grid, you have Trackmania players; automating the same exact thing, but for COMPLETELY different reasons. Trackmania having AI drive its tracks could be useful for any number of reasons: it could eke out milliseconds and show players tricks or strategies they may not see, they can be used to make more definitive gold/silver/bronze times with a more reasonable baseline of difficulty by tuning its abilities based on what medals you want it to achieve. I don't know where I'm going with this, I just found it funny that even in the virtual world, some people still have arguments against having AI drive cars despite demonstrably less stakes at play. This was a fascinating video and gave me a lot of food for thought. Thank you for all your hard work.

Very nice video 👏👏 Your next challenge is to create a ‘general intelligent’ driver that could finish an arbitrary map. Maybe by changing the map every few training steps the model could build more general understanding of trackmania driving.

really impressive work! it is often i come across ai playing video games and then being disappointed because the performance wasn’t great. This is a different case. Absolutely amazing job at getting the ai to work so well. I think it might lie in the rewards system being so well made(i am not an expert so please tell me if there’s other reasons for the ai’s performance) Props to all the people who worked on this, amazing job.

I'm impressed how well you got it working with visual input! It's interesting to see how people either overestimate or underestimate how well AI will perform. A balanced take is quite rare. As someone who's trained racing AI before, I'd love to see this applied to other prestigious tracks :D. I expect it would train a lot faster if you give it data like speed, the dot product of velocity with the car's forward vector, and one with the sideways vector. The latter really helps it learn to control the drifts in my experience.

@linesight-rl

7 ай бұрын

Thank you for your kind comment. I'm curious which kind of racing game you tried to apply RL to? As for the additional inputs you're suggesting, I did not take much time to explain this in the video but we are using this type of inputs. They are necessary along with the image to judge the car's dynamics, orientation, etc...

this is amazing well done

Wow congratulations!

This is really impressive. Great work ! I think you should experiment with a depth map input layer. Maybe by pluging in the DirectX's API. It might help the AI's better perceive it's position on the racing line and act accordingly.

This is great. I'd love to see it on a fullspeed map :)

this is so crazy, ggs

God I love ml so much. Fantastic video. Curious on the optimization of the relationship between framerate and resolution. Read some other comments noting that an increase in framerate would essentially be an increase in both, although I would love to see a simple experiment using a simple drift track with framerate and resolution control values for testing. Can’t believe tmnf community is still going so strong after 2 decades, makes me so nostalgic

amazing video!

Can you train an AI on the random map challenge? That way you'd get a more general AI. It might not perform as well on individual tracks, but overall it might do pretty good.

@MisterAssasine

8 ай бұрын

agree on that, or just take way more maps. Else you end up overfitting super hard

@hauthicus

8 ай бұрын

the wirtual ai

@vitorrodriguez3799

8 ай бұрын

I don't think that's how it works, the AI learns using a trial and error method, meaning the AI can only learn a map at a time. If you put it into a different map, it will be just as bad as it was in the start, and will have to learn everything again. At least that's what I understood from the video

@rikschaaf

8 ай бұрын

@@vitorrodriguez3799 It could work. It would probably require an extra layer and more neurons per layer, so it would take longer to train. Also there's a chance that it won't learn the maps well enough because of the complexity, but you don't know until you try.

@absolutecutie

8 ай бұрын

​@@vitorrodriguez3799because it's trained on a single map. But if it's made to be trained on random maps, it will learn probably slower but it will eventually be good on all sort of map (as long as it doesn't requires tricks the AI doesn't know)

This is so cool !

Incredible, it really is the perfect run, every drift endes exactly where it should, and I think the car accelerater a bit faster when “sliding forward”

Very cool, would like to see your improvements to the AI

Very impressive.

I would LOVE to see this AI become quick enough at learning new maps to compete in Cup of the Day. See if it can get division 1.

6:01 I like this diagram. It shows the meaning of training. Not only virtually, but also reality.

This is incredible. I hope to see it beat all human players soon!

I would love to see a write up with more detail about the training. I know it wouldn't make a good KZread video but I'd be absolutely fascinated. Obviously the next step is generalization. You could use Cup of the Day Maps as a potential training pool, though perhaps it would be best to ask the mappers.

Really good implementation! I have a couple suggestions for training. 1. Each epoch, create different intervals/ segments for the agent to learn (scale the reward according to the track interval length). This will help promote the agent’s ability to generalise along the entire track instead of exploiting a fast section of the track (common problem in RL). 2. Consider using an intrinsic curiosity module (ICM) for efficient exploration (also makes for good content to have a curious AI). 3. Train on different maps within one episode to improve general capabilities. As for the agent, consider using a continuous action space (steer values between 0-1) instead of discrete action space (steer left/ steer right). This could give it the precision it needs for world record. I understand from your ex platoon and your results you might be using Deep Q-Learning which doesn’t allow this, consider using an on-policy method like PPO or A3C. These improvements should make the agent more general and sample efficient, reducing training time and increasing capabilities.

Finding a simple cut would be an amazing AI achievement, because it has to divert from usual driving behavior using some complex information.

nice editing

Impressive!

Incredible Work! How good does this agent perform on other maps? Did it learn to drive good in trackmania overall or did it overfit on Hockolicious? How important was the graphic input compared to the engine input?

Just wow!

great vid and yeah 20 fps is definitely holding it back alot id imagine just a higher fps would allow you to beat the current world record

This is very super impressive. Since you asked whats next: no idea how feasibly this is but would be cool to get gold medal on every track in the campaign/career in TMNF, driving on different surface types with the same model might be interesting

@linesight-rl

8 ай бұрын

So, the objective would be to have one neural network that is trained on all maps at once, and able to reach gold medal on all these maps? It's definitely something we can try, I'm curious to see how it'll go.

@ethandavis7310

8 ай бұрын

@@linesight-rl man you gotta have a held-out dataset to show real performance

it would be nice to make an ai that is made to be good at generalization, eg, capable of doing new maps it hasn't yet seen fairly well/learn new maps quickly as it starts with good baseline "instincts", rather than it just potentially overfitting. But i understand that that would be much harder to do so its totally fair that it is the way it is. Incredibly impressive work! keep it up.

Cant wait for next Ai update

the AI is so good even though it's colour blind, has terrible eye sight and is lagging to boot

We are doomed. That's legit scary af and also impressive af. Damnnnn. WTF??? an AI drifts cleanly on hockolicious!??!??!.

As a guess, I think it could improve the result further if the ai can also steer values between 0 and full, or increase the inputs/time it can have

Good Job :) Further improvements could be using a pretraind vision transformer (e.g. fastvit) or cnn (e.g. mobileone) from the pytorch-image-models git and maybe (pre) train the model with rl from observation (since you can extract a lot of wr-runs and their inputs from the exchange)

Brilliant.

how explorative did you make the AI? if you have more compute time on your hands it would be interesting to see how the AI would do with higher exploration (and FPS, as you mentioned). loved the video+presentation btw :)

a few things that could improve the ai: - increasing fps like you said in the video (i think 30 fps is good) - let it have analog steering input (if that’s too complicated limit it to 0, 20, 40, 60, 80, 100% values) - maybe give color screenshots?

Great vid, doing everything i can to make youtube help you :)

I think it would be awesome to see it try to learn a track containing every different surface!

@linesight-rl

8 ай бұрын

If we're going in this direction, the AI has also improved its time :o) I think our last run on map5 went down to 2:02:08, but we didn't update the video.

This is lit

This really makes me want to attempt this with melee, much harder though since thats a player vs player game

Really just shows how exponential AI learning really is. My mom once said to me that AI will learn slowly at first, becoming smarter and smarter a little at a time. Then one day, it'll be as smart as a monkey, the next day a child, the next day an adult human. And the day it becomes as smart as an adult human, it'll shoot past humanity so fast in development that we'll basically become ants to it. Or so my mom says.

nice!

Pretty interesting

AI coming for our trackmania streamer jobs!!!

They way you train AI reminds me a lot of my elementary days

I have to wonder if a "two part" AI might be possible: A general "how to drive in Trackmania," portion and a "what do I need for this track?" portion. Training might involve cycling randomly through a couple dozen tracks, with the track specific portion being swapped in and out.

Man i cant even beat the AT wtf us this AI ! Really nice job this is amazing.

I would love to see if your AI could match the winner of the midori map. Especially if it can learn to use the unique turning needed on it to get that extra speed.

Super cool project! I have a question tough: Wouldn‘t the current speed in x, y and z be a useful input for the network to know? Especially for turning into corners I would assume that the static scene information is not enough. After all, when playing ourselves we get this information alongside the scene. I guess apart from feeding it as input, you could use a recurrent network that maybe is able to infer the speed from subsequent screenshots. I feel like using a recurrent network makes intuitive sense anyway. I would also guess that adding this input information would greatly help with generalizability. I worry that the model strongly overfitted to achieve this result. But to be fair, there‘s also a high degree of overfitting being done when we humans practice these runs. 😅 I‘d still be very curious how this model does on a different track.

Amazing

I wonder if it would become more capable if you rewarded it for its position throughout the run, instead of its own estimation of the final time (if I understand how it works anyway). Very interesting project, this deserves way more views

@linesight-rl

8 ай бұрын

THE tastyy ?!!!!! Your time on this map is really impressive, I hope my computer will beat it some day because I personnally struggle to just get the gold medal!

ptn c est stylé de ouf gg

I think this is great, somewhere between a TAS and a human run. Feels like it could be used another benchmark.

@ethandavis7310

8 ай бұрын

It basically is a TAS because he's performing on the same data that's being used for training. There's no generalization

I'd be really interested to see how much of the learning transfers from map to map. What if you took your flat trained model and used it on Hocko? Would it converge faster? Also, what is the "additional information" from the game engine going into the AI?

Lol beat Wirtual. SUBBED❤

this genuinely fucking scares the shit out of me

Time to train some ZRT Trackmania Cup maps

I know someone has probably suggested this, especially if youve been chatting with peeps in the TM community but, you can slow the game speed down rather than trying to increase your FPS. Theres been cheaters who've done that to set amazing times with human reflexes

Can you PLEASE upload a tutorial video, maybe even on a different tutorial channel? Your voice is extremely understandable and your explanations are amazing and make sense. Instead of watching 10 hours of people repeating themselves over and over again I would like to actually learn. Pleaseeeeee consider. thank u

I would love to see an ai trained with a more general learning method, so that it retains basic skills and only has to adapt to environments.

What I'd like to see (or try) is to have a selection of a N (e.g. 20) different, popular maps, and run learning in parallel and in batches, with random sequence of maps for different learning periods, and then test those against different maps and see if they can e.g. beat human record on 1st try :D

just...YES!

I'd like to see it on some fullspeed maps, I expect it to do well on those

Have you tried to change the camera settings? With cam 1 it sees more what lies ahead, but with cam 2 or 3 it can possibly resolve corners and curves better and thus cutting it more tightly