I haven't commented on a youtube video since 2017. But I have to, in the slim case that you actually read this comment Adrej! Please keep doing what you are doing! You are an absolute gem of an educator, and the millions of minds you are enlightening with each video will do great things that will compound and make the world a better place.

@AndrejKarpathy

Жыл бұрын

reminded of kzread.info/dash/bejne/dGx3l9WjmsvZo9I.html :D

@dhatrimukkamalla

Жыл бұрын

@@AndrejKarpathy you sure made YT comments section a better place lol.. Excellent videos, please keep them coming, or shall I say make more! Thank you!!

@NewGirlinCalgary

Жыл бұрын

@@AndrejKarpathy 🤣🤣🤣

@khalilsabri7978

Жыл бұрын

thanks for writing this comment for all of us ! please keep us these videos , as Minjune said , you're a gem of an educator !

@PatrickHoodDaniel

Жыл бұрын

@@AndrejKarpathy So lo no mo. Classic!

What a privilege to be learning from someone as accomplished as Andrej, all for free. The internet at its best🙏

@RalphDratman

11 ай бұрын

Just what this is -- a privilege indeed! We don't even have to pay tuition, or travel to Stanford.

@barjeshbarjesh8215

10 ай бұрын

I am not lucky; I am blessed!

@nickfruneaux5232

7 ай бұрын

absolutely!!!

@kamikaze9271

6 ай бұрын

So true!

@iNTERnazionaleNotizia589

4 ай бұрын

Hopefully KZread will be free FOREVER AND EVER, not like Medium or Towardsdatascience...

Update: I added some suggested exercises to the description of the video. imo learning requires in-person tinkering and work, watching a video is not enough. If you complete the exercises please feel free to link your work here. (+Feel free to suggest other good exercises!)

@AndrejKarpathy

Жыл бұрын

@@ibadrather oh. Please go to Discord then, linked in the description. Sorry :\

@ibadrather

Жыл бұрын

I don't know if this is a good place for Q&A but there is something I need to ask that I cant't wrap my head around. I was training the trigram language model and loss was less than for the bigram language model but the model was worse. I tried to generate a few names and I reliased I made a huge error in data preparation. The question I have is how big of an indicator is loss? Is loss the only thing that matters for model performance. I understand there are other metrics of model perfromance. I have actually faced something in my work. I am stabilizing a video using IMU sensor. And I am training a NN for camera pose estimation. For different architectures the lower loss models don not necessarily perform better. When our team looks at the stanilized video many times the model with higher loss generates a visually better stabilized video. I don't quiet understand this. That's why I am asking how much is loss the indicative of model performance. I don't expect you to answer this here but if you may talk abou this in your future lectures or somewhere else.

@OwenKraweki

Жыл бұрын

My loss for trigrams, count model, using all data for training was around 2.0931 and I was able to get close with NN approach. I'm not sure if the resulting names were better, but I wasn't able to generate the exactly same names with the count and NN approaches anymore (even using the same generator). Also I'm not sure how to best share/link my solution (I have the notebook on my local drive).

@sanesanyo

Жыл бұрын

I built the trigram model by concatenating the one hot encoded vector for the first two letters & feed them through a neuron & rest is the same. I think that is fine way to train a trigram model. Any views on that? I did attain a lower loss compared to bigram although the results are not significantly better.

@stanislavnikolov7423

Жыл бұрын

@@ibadrather Not an expert myself, but here’s how I would explain it: Coming up with a loss function is like coming up with a target you optimise for. Apparently your perception of how good a result is (your human brain loss function) differs from what you optimise your network toward. In that case you should come up with a better equation to match your gut feeling. Practical example. Let’s say you want to train a network that produces ice cream. Your loss function is the amount of sugar in the ice cream. The best network you train crushes the loss, but produces simple 100% sugar syrup. It does not have the texture and consistency of real ice cream. A different network may make great ice cream texturewise, but put less sugar in it, thus having worse loss. So, adjust your loss function to score for texture as well.

You're literally the best, man. These lessons are brilliant, hope you keep doing them. Thank u so much

@talhakaraca

Жыл бұрын

Seeing him back to education is great. Hope to see some computer vision lessons 👍👌

@HarishNarayanan

Жыл бұрын

@@talhakaraca If you search on this very site you will find truly clear lessons on computer vision from Andrej (from like 2016 or so)!

@talhakaraca

Жыл бұрын

@@HarishNarayanan thanks a lot. i found it 🙏

@HarishNarayanan

Жыл бұрын

@@talhakaraca You are very welcome. It was that lecture series that got me first informed and interested in deep learning.

The scale of impact these lectures will have is going to be enormous. Please keep doing them and thanks a lot Andrej.

@samot1808

Жыл бұрын

I am pretty confident that the impact of his CS231n course is bigger than even his work at Tesla. I know too many people working in machine learning that where introduced to the field by CS231n. It changed my life. Makes you wonder if he should just spend all his efforts to teaching. The impact is truly exponential.

@XorAlex

Жыл бұрын

Too many people are working on AI performance and not enough people are working on AI alignment. If this trend continues, the impact might be enormously negative.

@samot1808

Жыл бұрын

@@XorAlex please explain

@izybit

Жыл бұрын

@@samot1808 AI alignment and the AI control problem are aspects of how to build AI systems such that they will aid rather than harm their creators. it basically means that we are like kids playing with plutonium and it won't take much for someone to turn it into a bomb (on purpose or by mistake) and make everyone's life a living hell. All that leads to a need for more regulation and oversight of the really advanced AI models because otherwise we may end up with AI generators that can take a photo of you and create a video showing you killing babies, or worse, an AI that self-replicates and takes over entire systems leading to collapsed economies and countries (or, maybe, even something like the Terminator).

Never in my life have I found an educator like this. This is free gold.

The reason why this is such excellent teaching is because it's constructed bottom-up. It builds more abstract concepts using more concrete ones, generalizations follow concrete examples. At no point there is a situation in which the learner has to "just assume" something, which will "become clear later on" (in most instances when a teacher says it, it doesn't; it just causes people to desperately try to guess the required knowledge on their own to fill in the gaps, distracting from anything that follows, and producing mental discomfort). The bottom up approach produces pleasure from a series of little "a-ha" and "I agree" moments and a general trust in the teacher. I contrast this to the much worse fastai courses - in which there are lots of gaps and hand waving because of their top-down approach.

@deyjishnu

4 ай бұрын

This is exactly my experience as well. Well said.

I cannot imagine a better -- or kinder -- teacher. He is feeding his audience knowledge and understanding, in small delicious bites, without worrying about their level of prior knowledge. And he is smiling irrepresively all the time! Such a good person.

Absolutely insane levels of detail you are going into. This Series is invaluable for beginners in the field as well as for people like me, who are building own models all the time, but want to go back to basics from time to time to not get stuck in wrong assumptions learned from fast success with Keras :D I really hope you will continue this Series for quite a while! Thanks a lot, AI Grand Master Andrej!

Really incredible how you can explain clearly a complex subject only with raw material. Thanks a lot for the valuable knowledge

Another weekend watch! Epic to see these videos coming out! Thank you for all your efforts Andrei!

I love how you make the connections between the counting and gradient based approaches. Seeing the predictions from the gradient descent method were identical to the predictions from statistical probabilities from the counts was, for me, a big aha moment. Thank you so much for these videos Andrej. Seeing how you build things from the ground up to transformers will be fascinating!

@dansplain2393

Жыл бұрын

I’ve literally never had heard the logits are counts, softmax turns into probs, way of thinking before. Worth the ticket price alone!

Lex guided me here. I loved your micrograd tutorial. It brought back my A level calculus and reminded me of my Python skills from years back - all whilst teaching me the basics of neural networks. This tutorial is now putting things into practise with a real-world example. Please do more of these, as you're sure to get more people into the world of AI and ML. Python is such a powerful language for manipulating data and you explain it really well by building things up from a basic foundation into something that ends up being fairly complex.

I love your clear and practical way of explaining stuff, the code is so helpful in understanding the concepts. Thanks a lot Andrej!

Your intro to neural networks video is amazing, especially how you focused on the raw mathematical fundamentals rather than just implementation. I can tell this is going to be another banger.

Andrej, your videos are the clearest explanations of these topics I have ever seen. My hat off to you. I wish you have taught my ML and NLP classes in college. There's a huge difference between your ground-up, code-first approach and the usual dry, academic presentation of these topics. It also demonstrates the power of KZread as an educational tool. Thank you for your efforts!

The clarity from this video of all the fundamental concepts and how they connect blew my mind. Thank you!

Thank you for creating this - it's incredibly high-quality and I'm learning so so much from it! It's such a privilege to be able to learn from you.

This is one of the best educational series I have stumbled upon on YT in years! Thank you so much Andrej

This video is the goldmine. It's so intuitive and easy to understand. Even my grad classes could not squeeze this information over a semester-long course. Hats off and it's a privilege to be learning from the accomplished AI master and the best. Thank you for the efforts Andrej :).

What a pleasure to watch! love the fact there is no shortcut, even for what may seem easy. Everything is well explained and easy to follow. It is very nice to show us the little things to watch for.

I think minimal, simple-as-possible code implementations, talked through, are just about the best possible way to learn new concepts. All power to you Andrej, and long live these videos.

Phenomenal tutorial, thanks so much! I went through the entire thing and built it up from scratch and learned a ton!

A teacher that explains complex concepts both clearly and accurately. I must be dreaming. Thank you Mr. Karpathy.

Amazing videos! it's insane how detailed and intuitive these videos are. Thanks for making these.

What an incredible resource - thank you Andrej. I especially enjoyed the intuitive explanation of regularization, what a smooth way of relating it to the simple count-matrix

You're such a good teacher. Nice and steady pace of gradual buildup to get to the end result. Very aware of points where student might get lost. Also respectful of viewers time, always on topic. Even if I paid for this, I wouldn't expect this quality, can't believe I get to watch it for free. Thank you.

No one on youtube is producing such granular explanations of neural network operations. You have an incredible talent for teaching! Please keep producing this series, it is so refreshing to get such clear, first-principles content on something I'm passionate about from someone with a towering knowledge of the subject.

This is connecting so many dots for me. I really enjoy the combo of theory with practical tips for using the torch APIs.

I love how you explain every step and function so your tutorial is accessible for non-python programmers as well. Thank you.

What a sense of "knowledge satisfaction" I have after watching your video and working out the details as taught. THANK YOU Andrej.

Amazing delivery as always. Fact that he spent time explaining broadcast rules and some of the quirks of Keepdim shows how much knowledgeable he is and fact that he knows that most struggle with little things like that to get past what they need to do.

This is the first of your lessons I have watched and you really are one of the best teachers I've ever seen. Thank you for your efforts.

Thank you so much, Andrej, for the super helpful video. You are making learning complex things super fun and interesting, thank you so much

I love that he understood what a normal person wouldn’t understand and explained those parts.

Andrej, the elegance and simplicity of your code is beautiful and an example of the right way to write python

I just want to say, thank you Dr Karpathy, the way you explain concepts is just brilliant, you are making me fall in love with neural nets all over again

Andrej thanks for doing this. You can have a larger impact bringing ML to the masses and directly inspiring a new generation of engineers and developers than you could have managing work at Tesla.

You're getting me speechless the way you explain things and simplify concepts!!!!

You are an amazing lecturer! Thank you very much!

This lecture is well paced and introduces concepts one by one where later complex ones built on top of previous ones.

The density of information in these tutorials is hugeeeee.

I went through building micro grad 3~4 times, It took me a week to understand a good portion of that and now started with this. I am really looking forward to going through this series. Thanks for doing this Andrej, you are amazing.

Thank you for taking the time out of your busy days to teach others. This is a very kind act.

Another awesome breakdown, once again love how you take the time to help transfer intuition up and above implementation details.

Wow. Just WOW. Andrej you are simply too good! Thank you for sharing such valuable content on KZread, hands down the best one around.

Brillant, simple, complete, accurate, I have only compliments. Thank you very much for one of the best class I had in my life !

Hi Andrej, I heard two days ago (from a Lex Fridman podcast) that you were thinking in pursuing something related to education. I was surprised and very excited, wishing that it was true and accessible. Today, I ran into your KZread channel and I can't be happier, thanks a lot for doing this and for sharing your valuable knowledge! The lectures are incredible detailed and interesting. It's also very nice to see how you enjoy talking about these topics, that's very inspiring. Again, thank you!

I love how you take nn, and explain to us, not by already built in function in pytorch, but by how things works, then giving us what the equivelent lf it in pytorch

Fantastic! Thank you for sharing your many years of experience! You are a truly gifted educator!

Amazing explanations, Andrej. Thanks for sharing your knowledge in such a clear and enlightening way. Thank you soooo much! I'm really motivated thanks to you.

Andrej, i'm from Brazil and love ML and to code. I have tried several different classes with various teachers, but yours was by far the best. Simplicity with quality. Congratulations! I loved the class! Looking forward to taking the next ones. The best!.

Unbelievable lecture, Andrej 🙏 So many wonderful parallels. Thanks a lot for recording this and sharing it so freely with the world 🙂

The level of pedagogy is so so good here; I love that you start small and build up and I particularly love that you pointed out common pitfalls as you went. I am actually teaching a course where I was going to have to explain broadcasting this term, but I think I am just going to link my students to this video instead. Really excellent stuff! One small suggestion is to consider using Desmos instead of wolframalpha is you just want to show a simple function

You are incredible! This makes learning about ML so fun, your passion and knowledge really shine here. I’m a college student studying CS, and you lecture better than many professors. Not a knock on them though, props to you.

Thank you Andrej so much for these brilliant presentations. You explain each step perfectly! I have built my own simple neural network framework as an exercise to understand the gradient descent concepts and I am just starting to look at pytorch. I really like the autograd functionality, its very flexible as essentially the developer can use any combination of differentiable functions in the activation layer.

So excited to watch this! The tutorial about micrograd was great. Can’t wait to make my way through this one, loving the content :)

I’m an old-school software developer, revising machine learning for the first time since my undergrad studies. Back in the day we called them Markov Chains instead of Bigram Models. Thanks for the fantastic refresher!

Another absolutely fantastic tutorial. The detail is incredible and the explanations are so clear. For anyone watching this after me, I feel that the micrograd tutorial is absolutely essential to watch first if you want to really understand things from the ground up. Here, for example, when Andrej runs the loss.backward() function, you'll know exactly what's happening, because you do it manually in the first lesson. I feel that the transition from micrograd (where everything is built from first principles) to makemore (relying on the power of pytorch) leaves you with a suprisingly deep understand of the fundamentals of language modeling. Really superb.

So cool to see the equivalance between the manually calculated model and neural network model optimised with gradient descent. It's not quite the same output either. The regularization loss is required to get the two super close too. Pretty neat.

The music in this video is perfect. It really sets the mood and creates a powerful emotional connection.

Andrej, thank you for creating these video. They are the perfect balance of theory and practical implementation.

Thanks a lot for the videos. I was familiar with the concepts of basic machine learning but not with the actual workings of it. You really helped me get to the next level of understanding.

There are so many fantastic nuggets in these videos even for those already with substantial pytorch experience!

Can’t believe how easily you demystify these things!!! Can’t wait for the next one and the next one!

Thank you Andrej. I can't stress more how much I have benefited and felt inspired by this series. I'm a 40 yo father with a young kid. Work and being a parent have consumed lots of my time - I have always wanted to learn ML/neural network from the ground up but a lot of materials out there are just thick and dense and full of jargons. Coming from a math and actuarial background I had kind of expected myself to be able to pick up this knowledge without too much stumbling but seriously not until your videos did I finally feel so strongly interested and motivated in this subject. It's really fun learning from you and coding along with you - I'm leaving your lectures each time more energized than when it first started. You're such a great educator as many have said.

Thank you Andrej, this is absolutely the best hands-on coding neural nets & PyTorch tutorial. Special thanks for decoding cryptic PyTorch docs. Very, very useful!

This is such a valuable content! Keep it up, Andrej! Thank you for sharing your knowledge!

Two hours of pure clarity... soo addicting!!

Andrej’s way of explaining is exactly how I want things to be explained to me. It’s actually insane these high quality videos are free.

Mastery is ability to stay with fundamentals. Andrej derives the neural architecture FROM the counts based model! So the log counts, counts and probs are wrapped around the idea of how to get to probs similar to the counts model. Thus he explains why you need to log, why you need to normalizes, then introduces the name for it called softmax! What a way to teach. Brilliant master stroke is when he shows that the samples from the neural model exactly match the samples from the counts model. Wow, I would not have guessed it and many teachers might not have checked it. The connection between 'smoothing' and 'regularization' was also a nice touch. Teaching the new concepts in terms of the known so that there is always a way to think about new ideas rather than taking them as given. For instance the expected optimal loss of the neural model is what one would see in the counts model. Thanks Andrej! By the way one way to interpret the loss, is perplexity. What the number 2.47 says is that every character on average has typically about 2 or 3 characters that are more likely to follow it.

I absolutely love these entire episode, high quality content and very educational. Thanks a lot for doing this for the good of general public.

This is so awesome. Thanks Andrej for being so nice and sharing your knowledge.

10 mins into the video, I'm amazed, smiling and feeling as if I've cracked the code to life itself (with the dictionary of bigram counts). of course, I can't build anything with the level of knowledge I have currently but I sure can appreciate how it works in a much better manner. I always knew that things are predicted based on their occurrence in data but somehow seeing those counts(for eg. for first 10 words, of `('a', ''): 7`) makes it so glaringly obvious which no amount of imagination could've done for me. You are a scientist, researcher, high paid exec, knowledgeable, innovator but more than anything, you are the best teacher who can elucidate complex things in simple terms which then all make sense and seem obvious. And that requires not just mastery but a passion for the subject.

Amazing tutorial to explain the intricacies of various LM and ML concepts! Looking forward to the next LM video.

Please DONT STOP soing this! The world is so lucky to have you sharing this knowledge!

Andrej enseña de manera pedagógica y sencilla un tema muy complejo y además regala muchos tips invaluables de programación, python, Torch y cómo aproximarse a la solución de un problema. Tengo varios años aprendiendo ML, casi literalmente desde cero (no soy ingeniero, ni estadístico, ni programador), y estas lecciones me ordenaron muchas cosas en mi cabeza, (ahá moments, como comentó alguien en el hilo), entendí mucho mejor conceptos y procesos que antes apenas alcanzaba a intuir. De verdad es como abrir la AI y ver cómo es por dentro. Recomiendo ver los videos en orden, antes de este vi el de Micrograd y me pareció increíble entender todo. De verdad, mil gracias por este aporte Andrej.

Thank you for posting these! It's extremely valuable. The end result of the neural net wasn't all that anticlimatic, at least the longer "names" did differ slightly so it wasn't 100% the same weights as in the first case :)

I can not stress enough how much this videos are helping me. Thank u. A lot.

You definitely have a talent for education. Use it and you'll do so much good for so many people. Thank you Andrej!

Hehe, the bloopers at the end are priceless! I almost switched off the video when you said goodbye 😀

Thank you for this amazing video, really amazing and insightful.

You're lifting the lid on the black box and it feels like Im sitting on a perceptron and watching the algos make their changes, forward, and back. It has provided such a deeper understanding of the topics in the video. I have recommended it to my cohort of AI students, of which I am one, as supplementary learning. But to be honest, this is the way it should be taught. Excelllent job, Andrej.

The video series featured on your channel undoubtedly stands as the most consequential and intuitive material I have encountered. The depth of knowledge gained from these instructional materials is significant, and the way you've presented complex topics with such clarity is commendable. I find myself consistently recommending them to both friends and colleagues, as I truly believe the value they offer to any learner is unparalleled. The gratitude I feel for the work you've put into these videos is immense, as the concepts I've absorbed from your content have undoubtedly expanded my understanding and competence. This invaluable contribution to the field has benefited me tremendously, and I am certain it has equally enriched others' learning experiences.

Hi @andrej, I really like the summary that you did everytime you did something long. It really makes things super clear

Thank you so much Andrej . Amazing I am really enjoying every second of these series.

You're amazing! So glad to have you in this world and industry.

You're a really great communicator! Thank you for making this kind of content.

Thank you Andrej. Still practicing on this part and try to catch you up. Will use the knowledge gained from your generosity and make useful things to mankind .

These videos are absolutely gold! What a time to be alive

There's so much packed in there. I spent the whole day on this and got to the 20 minute mark, haha. Great teacher, thank you for this logical and practical approach.

This is SO good, thank you!

wow SUPER useful refresher 🎉

Doing this again to really solidify the fundamentals and github copilot is hilarious. It's seen this code so many times that if it is enabled you can't actually type everything out for yourself! It all comes rolling out (pretty much perfect, tweaked to my style and ready to run) after the first character or two. Amazing times. Got to say, whoa. This is so good. So sick of fumbling about with tensors and this is a masterclass for sure. Thank you thank you thank you.

new sub here. i started w/ "let's build gpt: from scratch, in code, spelled out". i learned lot, enjoyed coding along, appreciated the thoughtful explanations. i'm hooked & will be watching the makemore series. thank you very much sir for sharing your knowledge.

Fantastic!

Thank God you are pushing videos! Grateful 🤟

Amazing, this baby steps approach is so powerful to sediment the concepts.

Taught me how to do the Rubik’s cube, now teaching me neural networks, truly an amazing teacher!

This is a beautiful series! very well explained thanks A.