awesome presentation. watching the neural network in real time is hypnotizing.

@fenglema36

Жыл бұрын

Glad you enjoyed it!

@acuriousmind6217

Жыл бұрын

@@fenglema36 can you link the code in github ;) ?

@acuriousmind6217

Жыл бұрын

@@fenglema36 and btw awesome video earned a subscriber

@I.Odnamra

Жыл бұрын

It was that sounds for me. It spoke to me. Told me things. That it loved me.

@caturlifelive

Жыл бұрын

@@fenglema36 thanks for sharing

I legit didn't know there was a win condition on this game

@dennisengelen2517

Жыл бұрын

I thought the goal was just to score as much as points before you die.. 😅

@vanditsangal5018

Жыл бұрын

Not all such snake games had this condition.In some what he did at the end would result in regular elimination....

@fenglema36

Жыл бұрын

Great video :)

@vikasjha8199

Жыл бұрын

Same here, we just used to look at the highest score. Never to the end shown in this video.

@chaosagnostic

Жыл бұрын

Awesome video bro! @ezraanderson 🎉… @semituitive thanks for sharing my snake video here 😅

I always did the algorithm by hand, but never got fast enough eye-thumb coordination to finish it. This is so satisfying.

@fenglema36

Жыл бұрын

up, up, down, down, left, right, left, right, B, A and Start

@Shaun71290

Жыл бұрын

@@fenglema36 Konami code ftw

@fenglema36

Жыл бұрын

:)

@Rockyzach88

Жыл бұрын

If it's algorithm (unless greedy) then AI is not necessary right? Is the game solved?

@0818yukif

Жыл бұрын

グラディウス😅

I've completed this one in my teens using the pause button. Used exactly the same pattern. It was so satisfying like I won an olympic medal :-)

@Theocracy8

Жыл бұрын

Same here with Nokia old phone.

@garlicoshallots200

Жыл бұрын

Cap

@dragonflysword

Жыл бұрын

@Repent and believe in Jesus Christ It's much better this bot that keeps spamming the same exact message very on every thread. There's no creativity to try varying approaches nor even, I suspect, ability to relate to the topic, very convincing. Better grab my spot among those who have ceased thinking independently.

@dragonflysword

Жыл бұрын

The earlier majority of that exact pattern is by no means necessary, so that's pretty crazy.

@slowkr4v890

Жыл бұрын

@@dragonflysword way too risky and really crazy moves lol. But that's nothing for the AI I guess, a whole different story for a human 🙂

Imagine if the AI learns to drive vehicles and it starts to drive this way. Wait a minute...

@fenglema36

Жыл бұрын

Very witty :)

@indiankid8601

Жыл бұрын

Imagine if AI learns about self consciousness and it starts to revolt against humans. Wait a minute....

@mhmdcool1090

Жыл бұрын

@@indiankid8601 just throw a bucket of water on it tf?

@Manish_254

Жыл бұрын

Still it can't drive in india 😂

@fenglema36

Жыл бұрын

Only because there are cows on the road, and everyone knows snakes are afraid of cows.

Now I realize being patient and cycling back to the initial coords on map is better than just running toward the food.

A great demonstration of Neural Network AI... Some observations: 1. This AI probably didn't have the shortest path calculation in-built 2. As I could figure out, The basic logic that was built was not hitting the own body or the boundary 3. One of a good example of how neural network works (evaluating multiple options before making those 4 choices of Right,Left,Up,Down) As we are from Non Tech background, it would be great to have a small explanation of the code as well that went on to making this Algo..

@Inarka

Жыл бұрын

it is genetic algorithm. weight and bias of neural net are evaluated as gene with game score to make better decendants. states are determined by 8 identifing sensors in head, and 2 direction sensors in head and tail. identifing sensors sense that which exist (wall, snake, apple) in sensor direction, and direction sensors sense that which direction head and tail toward to. these informations are used as input of neural net. and actions are up, down, left, right. these are determined by output of neural net. This is a conjecture based on my knowledge, so it can be wrong.

@fenglema36

Ай бұрын

correct

It's incredible how quickly the computer can calculate all of those weights and biases in matter of milliseconds.

@sina_1239

Жыл бұрын

It takes a lot of time to train it though

@fenglema36

Жыл бұрын

About 8 hours of training

@joejohnson8966

Жыл бұрын

@@fenglema36 I played this game daily for maybe a decade. To think that only 8 hours can surpass what I did for so long is hella fast. It seems slow to us but it’s blazing speed. And AI is getting better and better.

@revimfadli4666

Жыл бұрын

​@@fenglema36 I wonder if evolution strategies could do it faster, without being more complex like Q learning

@Enthub47

Жыл бұрын

Perfect application of discrete maths graph

That was actually mesmerizing to watch. Thank you!

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

This is a good metaphor for how A.I can win better or win for certain, relative to what we can achieve.... yea the world is going to change big time! Edit: loved the work, you did great with this presentation.. you win too! congrats!

@fenglema36

Жыл бұрын

Thank you so much for the kind words :)

Fantastic visuals!

Didn't even know we could "win" Snake lol

@SeanKula

Жыл бұрын

Ikr

@LIONTAMER3D

Жыл бұрын

Yeah, we're all losers

@MikeSmith-ev3ef

Жыл бұрын

The woke left ruin everything

Thanks for showing the inputs of the network. What kind of learning algorithm is behind it?

Cool!🔥 Is this GA-trained?🤔 What type of variable of the neural network you use? (int? float?) And the neuron activation function?(logsig, linear, etc?) Do you use bias for the neurons? Thanks

@fenglema36

Жыл бұрын

thanks, Yes GA trained singles(floats), tanH and no bias.

I was waiting for the last moment to see how the win of snake game looks like. 😁

1:48 was impressive.

@fenglema36

Жыл бұрын

Thanks, please like, subscribe and share :)

And we do it just by, well, not even really thinking about it.

Yo that's wild. *cleans gun again

Awesome Vid!

@fenglema36

Жыл бұрын

Thanks

Awesome stuff bro, really appreciate you posting these exciting stuffs. Can you point me from where I can learn these ?

@fenglema36

Жыл бұрын

"towards data science" I read a lot of articles and watched a lot of you-tube videos on neural networks. andrew ng and Andrej Karpathy are a good place to start for youtube. I took a different approach and hand-wrote ever-thing in pascal, but there are a lot of python ML stuff out there that might be the easiest place to start, I am not very familiar with python so its hard for me to recommend stuff.

Chilling as fuck how AI learn so fast! Applications for this are great but dude it's chilling how a machine can do anything better than you and still learning so far

@a123b123c123d123c123

Жыл бұрын

That isn't the AI learning. it's the AI playing based on what it's learned.

awesome work man 👍 just curious, what crossover and mutation methods did you use?

@fenglema36

Жыл бұрын

nothing special but there is a simple rule very early on which will flush and kill generations early on if the starting weights don't procedure could results after 100 generation. Then I think there is a branch of 4 types of mutation that decay a little bit over time. Cross-mutation Inc weight Flip weight Inc weight toward other weight Think of a CPU architecture for predicting, if it makes an error it flushes the whole pipe line. I currently believe optimal starting weights will give you the optimal outcome of the network.

It's awesome but filling the space on an area like this can work as the only strategy needed. Also the biggest challenge for humans afair was the reaction time, and less so the entanglement with itself

I learned that I want to play snake again.

hello, a doubt I can only develop a neural network for specific things? what if i want it to do another function?

Imagine learning a robot pressing those finicky plastic buttons from those oldschool phones

This was great!

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

With known size of arena, initial position and without dynamical obstacles(except own tail) algorithm for win is actually just few loops. Any space filling curve or just scanning goes)

Wow man this is truly amazing!

@fenglema36

Жыл бұрын

Thanks, please like, subscribe and share :)

Beatiful 👏🙌

Now that's some high quality AI-asmr

Holy cow this ai is better at snake in 2 minutes than I was in 5 years… and now i feel like i should have known how to play! It’s so obvious now!

The winning sound is very satisfying

@fenglema36

Жыл бұрын

Thanks, I was concerned the sounds would be off putting... I feel like it fits the style/Theme though

I grew up playing this game and never knew that there was a winning condition.

Amazing job! How did you do the visualization of the network? Which tool did you use?

@fenglema36

Жыл бұрын

Everything was handwritten in delphi (pascal)

Awesome video, got a question though. I dont quite understand how the snake is learning in the video if theres only a single snake on screen (also no generation count), along with the fact that it seems to play almost perfectly from the start of the video. I ask since I made a flappy bird game with neuroevolution and it took a couple hundred generations to play perfectly.

@fenglema36

Жыл бұрын

This model is already trained. It trained for about 8 hours with a Genetic algorithm.

This is exciting as well as scary

I finished it once too… it was so satisfying

Thanks. So, is the source code available? If not, could you please explain the inputs? It is hard to understand their meaning from the video, namely what data from the box on the right should be sent to the inputs. Additionally, on the outputs: are they real world directions on the map or they are relative to current snake's head direction?

@a123b123c123d123c123

Жыл бұрын

Numbers 0 to 7 represent those 8 lines drawn out from the head of the snake. 0↖ 1⬆ 2↗ 3➡ 4↘ 5⬇ 6↙ 7⬅ For each of those numbers, there is a Distance, Apple, and Snake. Distance: The distance value indicates whether or not there is at least one open block of space in that direction relative to the head of the snake. Apple: This value indicates when the head of the snake is in line with the apple. Snake: This value indicates if there is a section of the snake directly next to the head of the snake in the direction specified by the number. Then you have numbers 8 and 9 which represent the direction that the head and tail are traveling. The values on the right are the direction in which the neural network will move the snake next. Those unlabeled nodes in between the left and right nodes are the hidden layers of the network and will be adjusted by some function allowing the neural network to learn.

@doctorshadow2482

Жыл бұрын

@@a123b123c123d123c123 , thank you. Now this is much clearer. From your nickname I can't be sure if you are the author of the video, anyway, since you have good insight, could I ask another question? The selection of input and output format is quite critical for Neural Network (NN) processing and getting correct results. Why exactly the binary representation for inputs were chosen? Why don't they use real numbers for distance? This looks more natural and can give more information to NN, does binary representation have its own benefits (like stronger input influence than gradual value)? Can you please, comment?

@a123b123c123d123c123

Жыл бұрын

@@doctorshadow2482 I am not the author so the info I give is just what I can tell or guess from watching the video. An integer value may be more useful for an AI but it also may be necessary. From what I can tell the Distance values just indicate whether or not the snake can move in a particular direction without hitting something. Also, binary values might suit the algorithm used better. Also the more complex the input the greater the processing power required.

@doctorshadow2482

Жыл бұрын

@@a123b123c123d123c123 , thank you for the prompt feedback. One comment, per my experience, in NN processing both binary and real inputs have the same processor power consumption, since the binary signal just converts into 0.0 and 1.0 of real number. So, this is just representation matter and this could affect the neurons triggering, but not the required processor power.

@a123b123c123d123c123

Жыл бұрын

@@doctorshadow2482that 0.0 and 1.0 are probabilities. It represents a certain weighting for a specific state. If you wanted to keep track of the actual distance using this method you would need a binary value for every distance increment. That would greatly increase the number of inputs and would increase the processing power required.

whats the lighting the cells up can just be ordinary resistors, if you keep the amping theoretical through the net and just amp it once at the end.

This is so satisfying to watch.

@fenglema36

Жыл бұрын

thank you so much, please like, share and subscribe!

That was a roller-coaster ride for my OCD

@fenglema36

Жыл бұрын

Some people find it soothing, other people find it annoying... its interesting how the mush in our heads interpret our senses

that was very satisfying and greatly presentated video.

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

What does the first layer represent. The distances in 8 directions i believe, and that more ?

Çok güzel olmuş tebrikler. ❤🎉

Looks far more successful than most NN snakes I've seen. What was the training algorithm? And what were the sound pitches based on?

@feywiild

Жыл бұрын

I think the sound pitch is based on the field, the head of the snake is on. When it moves up, the pitch goes down and when it moves down, the pitch goes up. Just my guess though.

@fenglema36

Жыл бұрын

The pitch is just a representation for the current grid location. 100 tones for this video. Its the only method I could think of to represent the network sound.

I never seen that game until the end Kinda mesmerizing

The sounds are very satisfying

@fenglema36

Ай бұрын

Thank you I like the sounds also :) some people hate them but thats okay

I programmed this as well, and realized very quickly how many inputs were needed to give the snake “vision” like you have done. Do you think it would have worked better to just have the contents of every block be the input? I wanted to go down that path but never followed through.

@fenglema36

Жыл бұрын

Pixels(Blocks) would be more difficult to get to work. In the snake game I wrote you can request "Pixels" or "Sensor" for the inputs of the Neural Network. When building this framework I focused on less data points. Meaning the inputs of 32 sensors vs 100 pixels(10x10) allowed it to train much faster. I would like to create an Actor Critic model which uses the inputs of Pixels, but have not been able to do it.

@MrAzureAzure

Жыл бұрын

​​@@MrMrmalobaidi I am curious as well. From the look of it, his inputs were 1x3 categorical array 0 distance - empty space or not 1 apple - the apple or not 2 snake - its skin or not on 8 directions around the snake's head (we got 24) concatenated to 1x4 directional array of snake's tail and head 0 up 1 right 2 left 3 down (24+8=32) Still impressive on how AI could learn to develop a strategy on given parameters

@fenglema36

Жыл бұрын

Bingo, that is a perfect explanation!

@chrisjfox8715

Жыл бұрын

I'm confused as to how a NN is the right tool for this

@fenglema36

Жыл бұрын

It probably not the best algorithm for this task.. But I hand wrote an NN framework and thought this would be a cool thing to try to do.

Holy shet, somehow a pleasure to watch.

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

I have a question. Obviously, there is an easy intuitive winning algorithm for snake. Does ai finds it optimal win-path? Or in this case simple algorithm is better?

@a123b123c123d123c123

Жыл бұрын

You could probably make a more efficient algorithm. This video isn't actually showing you the AI learning.

@fenglema36

Жыл бұрын

no it is not optimal. But I am trying to learn Neural Network and this seamed like a fun idea

Great video! How many iterations did it take until your model had the perfect weights? Can you also tell us which rewards you used? :)

@fenglema36

Ай бұрын

Far from perfect weights. It was a genetic algorithm with a fitness of steps + score. The goal of this was to finding the starting weights, then moves these to an actor critic model

The neural mind is very complex just for left/right up/down :o (So that means we have to appreciate to what we've been given, Because Remaking it is more complex than what You can say)

@fenglema36

Жыл бұрын

Feels like my daily routine Left, right, up and down.

Really impressive. Which tools have you used for visually representing the neural net and the snake board?

@fenglema36

Жыл бұрын

This is a handwritten neural network framework written by me in delphi (pascal). The NN and the snake game were written in 2020 Covid lock-down. Once returning to work I never got around to posting it. I dug up the framework a few months ago to work on convolution kernels, and decided to post this.

@yassineaguelmam8674

Жыл бұрын

@@fenglema36 genius

@fenglema36

Жыл бұрын

Thanks

Does The articulation points (yellow ones) work to majore the rest (snake's queue) to make the best way to prevent head accident? and the point on the eaten square parallely minimize the distançe between it and the head? الحمد لله.

@fenglema36

Жыл бұрын

There is no distance calculated, its just an On or Off switch if the ray-cast hit the snake, wall or apple.

Nice work for sure, but just to win the game you don’t need to have an AI algorithm, simple movement algorithm is quite easy to program to just win the game. What would be cool to see is that how different AI models to achieve winning state and which one do it better/faster. Because on the early stage of the game there still a lot movements to improve. Anyway, good work and it was interesting to watch!

@fenglema36

Жыл бұрын

Thank you. I Agree other algorithm would work, and this is overkill. I would like to do this project again based on different training methods, for example shortest path, longest path and Hamiltonian cycle.

@alexseveryanov1725

Жыл бұрын

​ @ezra Anderson Thank you for doing this, that would be very interesting to watch how differently other algorithms solve the same task. Once again, thank you for your work

@user-fh3wg8oj6q

Жыл бұрын

@@fenglema36 which data did the AI algorythm was training

@bramweinreder2346

Жыл бұрын

I actually think it was quite efficient. Making detours to avoid hitting its tail later, and finding shortcuts when applicable. If the AI learned this on its own then it's definitely ways ahead of a preprogrammed path with efficiency hacks. But if the path was programmed in as a bias, then it's still impressive, just not abstract machine thinking.

Breathtaking

@fenglema36

Ай бұрын

Than you so much, please like and share :)

I've only seen snake beatin one time back when I was small child in the 90s after that not even close

May I know what programming language and libraries did you use?

Wow it’s so cool so see NN animates

@fenglema36

Жыл бұрын

Thank you so much :) please like, subscribe and share :)

AI is only focused on the task, it doesn't have to think about holding the phone at the same time and pressing a button at the right time with the right force

Am I wrong or does the AI in top of winning go through a pattern that optimizes time pefectly?

This is PHENOMENAL!💫

@fenglema36

Жыл бұрын

thanks, please share and like!

@Zeddy272

Жыл бұрын

​@@fenglema36 already done!🙂

@fenglema36

Жыл бұрын

amazing,

Awesome

@fenglema36

Жыл бұрын

Thanks, please like, subscribe and share :)

It doesn't have to worry about hand eye coordination...great advantage for it.

@fenglema36

Жыл бұрын

or two left thumbs

Love the way they represented the neural net.

@fenglema36

Жыл бұрын

Thank you so much, alot people asked me for the code to draw then NN, but everthing was handwritten

@etienne8382

Жыл бұрын

@@fenglema36 always wonder if there has been any changes in the algorithm that calculates the weighting for each node the last 20 years or how the nodes are structured. Haven’t played around with it for years. But will always remain something cool to work with.

@fenglema36

Жыл бұрын

​@@etienne8382 to my knowledge networks are just becoming bigger as we can do a lot more compute now then 20 years ago. But Backdrop is still the most used for training networks..... I might be wrong as I am relatively new to A.I

Those are decently fast reactions, almost as good as mine.

I just realized that the completed game of snake is actually an ouroboros simulator.

@fenglema36

Жыл бұрын

googling "ouroboros "...... ..... ..... LOL Yep thats cool!

Most crucial aspect is missing which is training network. How and what was used to train the model.

@fenglema36

Жыл бұрын

Genetic algorithm... I will try to write a tutorial

Looks cool, what did you use for visualising?

@fenglema36

Жыл бұрын

Everything was hand written. The drawing is done in openGL with module I wrote for the snake game and the NN

Snake singularly

Such a great guy you are....I love it❤️ but how do you programmed it?

@fenglema36

Жыл бұрын

I am trying to work on a tutorial

Never realized how easy the last bit of snake is

@fenglema36

Жыл бұрын

I was also surprised by this. Apparently there is a very easy pattern.

@davidanderegg1232

Жыл бұрын

@@fenglema36 if the map was odd x odd it’d be harder. Also perfect turns are difficult for a real person

Great video, so what are your settings? Like batch size, max playback buffer size, gamma, learning rate, layer activation?

@fenglema36

Ай бұрын

This uses a genetic algorithm for training. I did start and actor critic version but it was never completed.

I’d be so nervous on those last few bits.

@fenglema36

Жыл бұрын

autopilot - and yeas its nerve racking

The snake eatting it's own tail is a win.

@fenglema36

Жыл бұрын

I found that humours also.. it reminded by of a puppy chasing his tail.

@driatrogenesis

Жыл бұрын

The ouroboros

Awesome work ✨. Could you please share the input featuresa and their meanings. What is Distance0, Distance1 represent?

@TylynGaming

Жыл бұрын

If i had to guess Distance0 is the snakes head and Distance1 is the apple

@fenglema36

Жыл бұрын

Every sensor is a binary switch. its on or off, snake, head or wall.

Does it use a pure neural network, or are there additional algorithmic rules coded? Like avoiding obstacles, etc.

@fenglema36

Жыл бұрын

The neural network is trained with a genetic algorithm. The generation with the best fitness is who ever takes the most steps and gets the most apples. I think the only rule on the fitness test is: you have 100 steps to collect and apple, then the steps reset. So he theory is longest path to the apple.

@grantleland6913

Жыл бұрын

@ezra anderson I think it would be interesting to use a Q learning approach. Like you mentioned an Actor Critic network would be a cool algorithm to implement for snake. I’m assuming you would be able to get results much faster from a Q algorithm trained by Adam than a genetic algorithm especially since you have a well defined reward. Figuring out the inputs for this would be more tricky since NNs do well with continuous state spaces and snake is more of a discrete state space. Perhaps using all 100 block values as an input could work well for this type.

I don't know much about ML . But I did a CNN Image classifier. Sry If this comes as a dumb question but does the snake know where the next bite(red thing) is going to spawn . I mean aren't there any configurations in which whatever it does it needs to bite itself.

@a123b123c123d123c123

Жыл бұрын

The Snake detects the appl/red thing when it moves in line with it. Look at the labels on the input layer (left-hand side). When the head lines up with the apple the corresponding Apple node lightes up.

@fenglema36

Жыл бұрын

When training it bites itself a lot until it learns not to, the apples are spawned at random

I love your video.

@fenglema36

Жыл бұрын

Thank you, please like subscribe and share :)

Great work! What's the input data set that was used?

Awesome bruh

@fenglema36

Жыл бұрын

Thanks, please like, subscribe and share :)

I’ve got anxiety watching it chase it’s tail 🤣🤣

This is so cool, what is the reward mechanism for your agent?

@fenglema36

Жыл бұрын

it's really simple fitness test on the agent that lives the longest moves * appless

Bro the algorithm ai used is kinda next level zero error

That was fun!

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

This is how my dad play video game in his childhood

Amazing 😮

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

Переживал за эту змейку больше чем за свою жизнь

OMG … this was so satisfying !!!

@fenglema36

Жыл бұрын

Thanks you so much I also find it satisfying. please like, subscribe and share :)

Wow. This is mind blowing.

@fenglema36

Жыл бұрын

Thanks, please like share and comment

Damn, AI accomplished what I always tried to achieve.

How do you visualize the neural net i tried to do that but i counldnt find any resoursces. Does anyone know

@fenglema36

Жыл бұрын

This neural network framework was all handwritten, I coded the draw module

I went up to like, 80% phase of that) GG

That's really awesome

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

Beautiful

@fenglema36

Жыл бұрын

Thank you so much 😊 please like, subscribe and share :)

Great !!! Man I want to learn this Any suggestions where I can start from ??

@fenglema36

Жыл бұрын

CodeEmporium kzread.info/dron/5_6ZD6s8klmMu9TXEB_1IA.html Andrej Karpathy kzread.info/dron/Pk8m_r6fkUSYmvgCBwq-sw.html Andrew NG kzread.info/dash/bejne/goSA0dJtfJXLd84.html

Got you bro - AI is different than a simple pattern.. Means the snake only curls up to down which should be sideways and also unpredictable.. yess "unpredictable"

what software you used to code the AI and monitor it's decision?

@fenglema36

Жыл бұрын

Everything was hand written in delphi (pascal)

Wanna see more progress. Seems like it’s pretty sporadic early on trying to find the cube. At the end it knows the proper movement. Bet with more time I could be more precise early game

@fenglema36

Жыл бұрын

interesting, its trained on a Genetic algorithm reward function based on longest path(more steps are better).... so at the end after its collected all 100 apples, if you change the reward function to shortest path(less steps are better).. I wonder if ti would be more efficient. hmm very interesting.