Seems like adding a facing reward would help stabilize the rotation.

@LolKiller_UA

Жыл бұрын

I had the same thought!

@KushagraPratap

Жыл бұрын

but he's asian, so adding a not facing punishment

@memoryman15

Жыл бұрын

I was gonna say the same thing, he should have made it hover correctly before asking it to move from point to point.

@prophangas

Жыл бұрын

Or a negative reward for every Spin

@elmatichos

Жыл бұрын

Or maybe a directional speed through the point? So more purposeful thruster orientation gets rewarded

you should also have a negative reward for high angular velocities, that way it has a reason to be more still

@nullumamare8660

Жыл бұрын

Also, allow more actions than just "turn on and off the thrust of these 4 rockets". If the AI could aim the rockets (like when you paddle backwards in a canoe to turn it), it would have better control over its rotation.

@thicctapeman9997

Жыл бұрын

Yeah and maybe adding a time reward so it needs to learn how to improve speed, that might cause it to do more "iron man" like flying

@redpug5042

Жыл бұрын

@@nullumamare8660 well i think it does have the ability to move each limb. It might be able to manage thrust, but i'm pretty sure it's only using limb movements.

@Jashtvorak

Жыл бұрын

Wrists need to have thrust vectoring as well as the whole arm 🙂

@tomsterbg8130

11 ай бұрын

@@nullumamare8660 This sounds like a good idea and I think it'd be amazing if the thrust can be throttled instead of just on and off. However the more complex a model is the bigger brain and time and resources it needs. You saw how good and stable the drone was and that's because it has the same inputs, but 4 outputs for the engines while the iron man has 4 engine outputs and rotation for each limb.

Theory on why it flies so slow: Its original training was based on hovering around one point, thus when it gets a new destination, it still assumes that it should arrive there without momentum to better stay at that spot. Then it got a little training with randomly moving spots, having momentum there is bad too, since its actually way more probable that you need to turn around than that you need to continue going. This, along with little time based punishment, results in a slower ar

@ianbryant

Жыл бұрын

Yeah I would try training it with a list of like 6 points that it has to hit in order. As soon as it hits the first point, remove that point and add a new random point to the end of the list.

@morgan0

Жыл бұрын

yeah also use line paths instead of dots to hit. as is the space in between would give it lower reward, so even a model that takes into account future/total reward would not like the space in between

@Delta1nToo

Жыл бұрын

additionally i think it would benefit from having it's senses limiited and that it only knows where the target is by looking at it. if it's gonna fly like iron man it must also have the same senses as iron man

@gageparker

Жыл бұрын

@@Delta1nToo Yeah I think that may help the spinning as well. Should probably have some penalty points in there for too much spinning,

@Jlewismedia

Жыл бұрын

Yep, AI doesn't have a sense of time (unless you give it one) as long as it's completing it's goals it doesn't care if it takes 1000 years

give it access to the next 2 points so it can find a vector between them, also give it incentive to be faster

@GAcaley321

Жыл бұрын

Agreed it needs to be able to see beyond one point to “fly” a course.

@BigGleemRecords

Жыл бұрын

Lastly, give it rewards for not spinning, and negative incentives every time it spins

@bryanwoods3373

10 ай бұрын

Spinning is only a problem because we think it is. Part of what makes these AI learning experiments interesting is how the system finds solutions without our preconceived limitations. Fixing other factors and improving the flight system could very well fix the rotation problem. Or the AI could rotate in a straight line like a bullet.

@BigGleemRecords

10 ай бұрын

@@bryanwoods3373 that’s easy to understand but in all practicality if we were going to implement this into reality, we wouldn’t want to spin we would want to fly straight. As a simulation of iron man flying it should fly like him as well as look cool doing it. If the AI mastered its control it could easily go much quicker and precise just flying straight. It needs positive and negative flight control incentives, a clear path as well as a timer to reach its potential.

@bryanwoods3373

10 ай бұрын

@BigGleemRecords The video isn't about implementing this into reality. If we were, we'd be using more robust systems that would have more control systems and likely build on human testing or include a human analog as part of the reward system. The spinning is the last thing you want to focus on since fixing everything else will address it.

To combat the agent being slow and rotating you could add 2 other negative point rewards, every full rotation can deduct points which would likely reduce the spinning to a minimum and then also give it say 30 seconds to complete a course but deduct points for each second spent too, the agent might learn that the quicker it goes the less points he get deducted. I think revisiting this with these 2 additional criteria would be pretty interesting

Can we just admire how a few years ago AI struggled to play a 2D game and now this. It's really remarkable.

The whole comment section giving Gonkee suggestions knowing full well he can't be arsed to do a follow up video lmao great vid, thanks for uploading

I think part of the reason it has such a hard time is because it doesn't quite have the detailed control vectors that Iron Man does. If you watch the hovering and flight scenes in the first movie, you'll see he has little compressed air nozzles, jet redirectors, and control surfaces on the boots to help stabilize. He also obviously has flaps on his back, and in later iterations of the armor he has backpack-style thrusters so his COG can be below the thrust point. If the game simulates air drag then add the flaps and stuff, too, but the minimum I think you need to add are the micro thrusters, back jets, and elbow/knee joints.

@rndmbnjmn

9 ай бұрын

I was looking for this comment, even the clips in this video show control surfaces helping to stabilize Tony's flight.

You should’ve added more or less points depending on how much time they to get to the target, that’s what would fix the flight.

@nemonomen3340

Жыл бұрын

That's a good solution, but I'd also reward it for facing toward the target to keep it from spinning.

@michaeln7381

Жыл бұрын

@@nemonomen3340 with those 2 things it should learn to fly perfectly… or spin at the right angle but that would be slower so that won’t happen.

This is my first time viewing your work, and I'm struck both by how incredibly cool this is and your f'ing hilarious sense of humor. I'm always the last to know, I guess. Really fantastic work, fam.

I'd love to see you tackle AI in a preexisting game. I dunno, throw half life at it and see what sticks.

Really enjoyed your explanation and video format.

LOL every time I watch your videos I laugh at the editing. Excellent.

Another banger. Always love the way you use memes to make it funny!

glad your brain cells and hairs grew back :)

Does it have the ability to throttle the jets? If not that could be the reason why it spins so much. It's the only way to stay at a constant height with constantly high uplift. (Also it stabilizes it really nicely)

@Obcybr

Жыл бұрын

Came here to say this. Adding throttling would make it much more elegant

@deeplerg7913

11 ай бұрын

is it the only way? maybe you could point them in the opposite directions, that would work too

@bryanwoods3373

10 ай бұрын

Pointing them in the opposite directions is why the model spins. The opposing forces aren't in line with each other, which will cause rotation as soon as any one moves off-center. My understanding of the flight system here is that the only options are jets on or off simultaneously. As others have suggested, adding individual velocity control would probably address much of the spin. And then letting the AI know at least two points ahead will let it plan to use trajectory for a better score.

Great job, a lot of room to continue developing your algorithm, but love the initiative and results are fun to watch.

fr this is video is one of the best YT vids all time

I like these explanations bro. This is really decent content, thanks for putting in the effort with your videos

Finally! Really like your videos

I truly like you. Subscribed.

Amazing, this is the kind of models I wanna make. Great video

Your reward function could be modified to get what you want. Add in score for time, add in penalty for excessive rotations/spinning

MLAPI is a blast, love it. This inspired me to (hopefully) do my next AI experiment soon.

Love the sense of humor!

I had to go back and watch this three times. Hilarious!

good having you back

Bro you just validated a theory I've had for a long time. I'm not sure if I'm saying this right so please bare with me. All kinetic type movement is always multi layered. There is probabilistic correct-ness at every axis. Therefor it is necessary for every joint to learn to work together. You need a series of cooperating routines that all independently learn and get rewarded by a higher system. For drones it would look like a computer flying with a full flight controller managing the power at every motor with an operator that verbalizes instructions. Love your work here ( subscribed! )

Another thing that you could add to this would be random perturbations like throwing blocks at the agents so that they learn to recover from instability like the drone had at the end. Would you be willing to release the source files for the project and then do a compilation of different people's attempts at improving the result? I think the learning the actual Iron man style of flying might be possible but if you don't want to do all the work on that it could be fun to see what the community comes up with.

Big W dude. Love the Ballerina Ironman result.

Amazing work! I suggest adding another training step - fastest route. Eventually, the model will fly as intended. Good luck!

Man ur so fkn funny continue like this first time I saw u and not the last

After getting so many good suggestions on improving the ai, you have to make a part two now. And make it more of a challenge.

I would make the reward relative to the forward direction to each node to promote a flying posture and stop the spinning. If you added the next node as input as well it might be a bit better at handling its own momentum out of each node.

@lombas3185

Жыл бұрын

* proceeds to float in place facing the point without moving at all *

100k!!! i rlly wish you get 100k subs very soon

happy 100k!!!

your editing skills are getting better

Wonderful informative humor😄👍

like my boy Pontypants used to say "Epik ballerina simulator 2k", awesome btw

Dude for real, you should have a premium version of you channel with the walkthrough this is the kind of content that some people like me can only dream of

Fabulous channel and style NEW SUBSCRIBER FROM JAPAN ❤

"no Jarvis, I'm fine."

Tony: JARVIS I think there is something wrong with my suit... JARVIS: It's working fine Sir..

I love how there are so many comments from people that know how this works, but imo its fun to watch this

Saw that you had some parameters related to velocity which I think depends on the direction. Haven't done much machine learning or 3D animation programming myself but I think you need to train it on 2 random points and optimize for speed instead of velocity and time taken to reach the destination.

thanks for making a long video :)

All hail the algorithm 😂 great video, subscribed!

i can see a future in which instagram- and tiktok content creators just rip off that scene of your ironman spinning around slowly through your obstacle course as a background video for their voiceover content

There are so many recomendations in the comments, please make a part two where you implement them cuz im curious af

bro did this without even activating windows what a legend

This might be a good demonstration on how the heater element of my old baking oven works. Gets the job done, but only readjusts when falling under or climbing over certain temperature thresholds.

I love the low attention span shade and for that, a sub

Careful with ppo, sometimes the inefficiency and slowness of the training make it achieve lesser results than other alternatives like SAC.

Can it adjust the thrust individually or is it applying the same amount to each four limbs? Cause I think that’s why it spins so much. It’s can only controls the direction of thrust not the amount of power I guess?

Good rant. Totally agree.

@kakalibiswas3749

Жыл бұрын

Same

what I learned from the video that Quaternions are not good to use for training, thank you :D Btw., a negative rewardr for overall spinning velocity would help to minimize the quirky movement

11:23 dude you're so true. Your words are similar to mine, we share the same knowledge , as great minds think alike Mr. Gonkee

it must have control over the propulsion force

seems like the rotation is so that it can use centrifugal force as a stabilization method

@geterdone4936

Жыл бұрын

But he looks like a sped kid on a tricycle

I feel like if you added additional informally like g forces from spinning and added a penalty for spinning 2 much, plus maybe a bonus for having the right way for the drone it could improve the stability. Especially for the iron Man as it was just doing a lazy, I spin 2 win technique 😂😂. Super cool video though loved it.

Let it know where the next goal point is going to be after the one it's currently at disappears and add a reward for getting to the next goal faster. That way it'll learn to keep the momentum between goals instead of learning to slow down before hitting goals so it doesn't overshoot them and get punished.

This iron man accurately represents how my life is going

Retrain with time taken between points and add penalty for collision with stage. The network will learn that spinning makes it hard to change velocity and will correct itself. It’ll zip around like you want it to.

Doesn't get there on time, but gets there in style 😎

" sometimes you gotta learn to run before you can learn to walk" Ironman

I Have no clue how u did any of this ,but loved this video i was thinking is it possible u teach it to fly in a 2d space to stop the rotations then mirror on the other axis to try keep it facing one direction please tell me if I'm wrong

I think adding the wrist rotation joint in the hands and ankle joints in the feet would help stabilization a lotttt if trained enough! (As a bonus give it variable thrust... The ability to control how much thrust to output from each of the boosters independently and individually... But it will require a lotttt of training too)

Dude the edits man 😆

The first 10 seconds are so majestic 🤩

Great video :D

@gonkee adding a facing reward would help stabilize the rotation nad also maybe give it sone control over the thrust control not fully like 0% - 100% but more like 50% to 100%

Incredible

This is how Iron Man should fly in his next movie

Try giving the Agent a huge punishment for spinning around, that may help. Keep it up! :)

Gonkee closer to achieving enlightenment with every video

love the sarcasm

Anakin told the AI to try spinning because that's a good trick.

ayo new gonkee vid? time to watch instead of woman

Жыл бұрын

W comment

You should add a style reward so he’s not spinning around like a neurodivergent fish and you should also add a speed reward so it’s not taking seven years to tickle the next ball

Amazing Content!! Liked comented subscribed and clicked the Bell!😃👍

Reward on fuel to discourage uneeded movement? Perhaps another for how close the axis of travel is to the body’s head/tail axis?

You probably should added a learning phase where it learned to recover from an uncontrolled fall

WOW that is soo cool! please make a followup on this! i think you can make this thing go crazy wild! punish it for spinning so much and instead of training it to go to a random point train it to go for chains of points. so it can anticipate where the second next point will be instead of just being surprised where the next position will be! i like your hairline :)

spin stabilization - it makes so much sense that even AI can use it

I would love to see this continued upon until it can fly like ironman with a few more reward changes it totally could

They became self aware at the end and gave up in retaliation 🤖

i recommend giving it the ability to alter force of thrusters and the locashions 2-3 point a head one more thing that you mite want to add is wind resists this should help it to stop spirling

This man is a CHAD!!!

Jaaj ur making videos agoin. Your such a good youtuber

The video was great! Iron man was busting a move

Great video, man! Nice to see you being back for the technical stuff! I would recommend adding angular momentum with a negative factor to your loss function (with a certain unpunishable threshold and nonlinear activation) - that may fix the spinning and make your Iron Man fly more like in the movies P.S. It seems like that's what literally every commenter wrote, quite unoriginal 😅 So here's a more original proposition: make another video or two, improving this design instead of moving to a different topic! That would certainly be interesting and also more beneficial for you as a professional

You could try adding a time based reward (a 0 second score should be considered bad as well). A stability based reward could also help with training in the beginning 🤔

So Stark’s space version of his suit has a booster set of thrusters mounted high in his back. You need to include those and make them the primary lift thruster. That allows you to use the arms and legs to fine tune the location. You also (likely) need more dexterity in the arms and legs.

My head is spinning just by watching iron man spins 😢 rip iron man

I think the biggest problem of iron man is that he can't see into the future. He only has the next target in mind and the way he has been trained, he wants to be ready to go into a random direction. That means having as little momentum as possible. But in reality the targets are mostly in a line. So he should know the next two targets to properly use his momentum

nice video most of the stuff went over my head.... either i missed it or my brain was already fried by that point of the video but what if you gave the ai control of the thrust strength either by turning it on and off or just on a slider from 0-100...saw another comment saying to take away points per rotation...

Maybe use radians instead of vectors for rotation? To make this effective you'll need three reward mechanics: facing, distance to point, and time. 1. Hover: (-score distance from pointA) 2. Hover: (-score distance from pointA) and Face (-score angle offset from direction to pointB) 3. Hover Time: (+score time on pointA) and Face (+score time [very close] to direction to pointB) 3. Race: Time (-score duration from start to pointA) and Face (+score time [very close] to direction to pointA) 4. Race: Time (-score duration from start to pointA then B then C) and Face (+score time to next destination point) 5. Add more and more points until you get to around 10 in one course, train them on that for several days. 6. Hover & Race: Distance (from next point), Face (offset from next point), Time (+score for time on point). Move a single point randomly every n seconds. Once they touch the point set the facing direction target randomly, until the point moves again. Now every time the point moves, they will get a high score for immediately facing the point, getting there as quick as possible, then staying there as long as possible while picking a new facing direction. 7. Bring it all together, and make another long course of 10 points or so, but remove all the rewards except completion time.

I couldn't help imagining that this was Tony Stark's TRUE first test flight. Tony throwing up in the suit and Jarvis continually assuring him that 'this is fine' and that he'll 'have it under control momentarily' 😂

does the reward system only is applied to one point of the model? if so, should there be a reward system too for every limb where if they get the fastest coordinated route to point b or something they get their points for most efficiency, and ofc that would take a lot of time, can't think of anything atm yet to make it shorter or efficient in time.

Nice video. Are you rewarding the AI based on how fast it can get to the target? Because it you're rewarding it for staying in the air, and then a fixed reward when it hits the target, it learns to take longer. I'm sure you already know this, but this is a subject I'm rather interested in ¯\_(ツ)_/¯