Word Embedding in PyTorch + Lightning
Word embedding is the first step in lots of neural networks, including Transformers (like ChatGPT) and other state of the art models. Here we learn how to code a stand alone word embedding network from scratch and with nn.Linear. We then learn how to load and use pre-trained word embedding values with nn.Embedding.
NOTE: This StatQuest assumes that you are already familiar with Word Embedding, if not, check out the 'Quest: • Word Embedding and Wor...
If you'd like to support StatQuest, please consider...
Patreon: / statquest
...or...
KZread Membership: / @statquest
...buying my book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
statquest.org/statquest-store/
...or just donating to StatQuest!
paypal: www.paypal.me/statquest
venmo: @JoshStarmer
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
1:53 Importing modules
2:48 Encoding the training data
6:55 Word Embedding from scratch
16:54 Graphing the embedding values
21:17 Printing out predicted words
20:37 Word Embedding with nn.Linear
28:12 Loading and using pre-trained Embedding values with nn.Embedding
#StatQuest #neuralnetworks #transformers
Пікірлер: 75
Here's the code: lightning.ai/lightning-ai/studios/statquest-word-embedding-with-pytorch-lightning?view=public§ion=all To learn more about Lightning: lightning.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@loserc1854
2 ай бұрын
waiting for ur next book
The best part of this example is the way the first example is written with single dimension arrays for all of the parameter values. That makes the matrix operations explicit and more clear even to an experienced developer. I recommend writing and debugging the code from watching the video. There are small things that you might get wrong that will teach you something.
@statquest
4 ай бұрын
Thank you! I'm glad you liked the example.
I bought The StatQuest Illustrated Guide to Machine Learning, and it's absolutely amazing, same with every single one of your videos! I highly recommend that book to anybody who is interested in learning about this stuff! Thank you thank you thank you so much for taking the time to put together the content you make, I can't believe how well you paint this stuff into perspective! You're a freaking awesome person Josh, I'm still absolutely elated that I stumbled across this channel! I never would have expected to be so consumed with learning this and everything about this!
@statquest
7 ай бұрын
Thank you very much! I'm really glad you enjoy my videos and my book! Thank you! :)
Hi Josh, I just want to say that you are an absolutel godsend to the ML and DS community. I have been following your content since 2020 and you have never let me down. I hit a block with PyTorch code implementation and your videos simply untangled a lot of my mental knots. Keep up the great work!
@statquest
3 ай бұрын
Awesome! Thank you!
Josh- you are a Master in making the difficult comprehensible!
@statquest
4 ай бұрын
Wow, thanks!
I like your videos. Both the slides and explanations are very detailed and clear. I cherish them every time I watch them. It is a good learning video. thank you very much.
@statquest
4 ай бұрын
Thank you very much! :)
Just finished the Deep Learning playlist. Thank you so much for this great work.
@statquest
6 ай бұрын
BAM! Thank you very much! You deserve an award!
Josh you are one of a kind!! Thank you for all the content!
@statquest
6 ай бұрын
Thank you!
Great video, a great way to study. Thank you.
@statquest
7 ай бұрын
Thanks!
Thing is; i always look forward to new videos from you, cause i know it would be awesome as always. I would want to see more videos on computer vision tho, just hope you'd consider this. Thanks
@statquest
7 ай бұрын
I'll keep that in mind.
I too, love your work and specially appreciate your playful style and all the movie references. ;)) Thanks for your videos, all of them.
@statquest
7 ай бұрын
Thank you very much!
You rock. I love your work. Salute from Brazil.
@statquest
7 ай бұрын
Muito obrigado! :)
Yayy !! I was wanting a video like this !!!
@statquest
7 ай бұрын
BAM! :)
Great video as always
@statquest
7 ай бұрын
Thanks again!
Thank you!
@statquest
7 ай бұрын
:)
StatQuest is so inspiring! Please consider making a 'Quest about Geometric deep learning. What do you think?
@statquest
6 ай бұрын
I'll keep that in mind.
This is really great 🎉🎉🎉. Can you please continue this pytorch + lightning series
@statquest
6 ай бұрын
Yep! Working on it right now.
@kisholoymukherjee
6 ай бұрын
And hope you will also add them to the Deep Learning Playlist. Keep rocking (and BAM-ing!) @@statquest
@statquest
6 ай бұрын
@@kisholoymukherjee Will do!
StatQuest Time!!
@statquest
7 ай бұрын
BAM! :)
Superrrb Awesome Fantastic video
@statquest
7 ай бұрын
Thanks!
Hi! I also bought The StatQuest Illustrated Guide to Machine Learning - It's awesome! Triple BAM! I think there might be a small error with the formula on page 135 (Naive Bayes: FAQ Part 3) though, FYI! Thanks again!
@statquest
6 ай бұрын
This is a known error in the book. All of them are listed here: statquest.org/sigml-errata/
Thank you again for the quality content. 🔥. Btw is there any plans for teaching about fine-tuning Language models. I heard this paper (Universal Language Model Fine-tuning for Text Classification) is of higher relevance and backbone for many state of the art solutions.
@statquest
6 ай бұрын
I'm working on one.
Hi josh Starmer, after learning ML, whether this series is enough for a s beginner to learn deep learning? and do any additional things to know other than this series for deep learning?
@statquest
5 ай бұрын
It's a great way to start.
as a related thing, finding closest things on a map/grid, without making a cell division system, sort all dimensions then check if they collide, like spheres or aabb bounding boxes, n log n time complexity, as opposed to n^2 time complexity, similar to grid cell partitioning of data, for clustering, or k-nearest clustering, but just finds closest at range, any number, very fast. should be applicable directly in high dimensionality complex data analysis, dimensionality reduction, covariance matrices, yep its designed to be fast
Plz start teaching yolov8 which is used for object detection, segmentation, classification ... Every problem will be solved plz plz
@statquest
7 ай бұрын
I'll definitely keep that in mind! Keep reminding me on future "in PyTorch + Lightning" videos.
@vigneshvicky6720
7 ай бұрын
@@statquest sure sir but try to do it from scratch because everyone are using build in one so many of them dont know what is going inside. I would like to learn it from scratch so that I can finetune architecture myself . Do it as soon as possible🙏
Do u guys have an education I can get somewhere? I saw u guys had a probability basics playlist but do u have some type of course to take me through every single subtopic of stats?!
@statquest
7 ай бұрын
Not yet.
👍So, these are word embeddings. Do you know/think that sentence embeddings and text embeddings differ much from this process?
@statquest
7 ай бұрын
I'm pretty sure they are the same, but I'm not certain.
The original work [1] uses a multidimensional vector for each word instead of a unique real value. For instance, Troll2 = [,025, 0,735, 0,256, 0,145], in this case, four dimensionality. In the paper, the authors use a matrix instead of a neuron. Question: using neurons, how would you increase the word dimensionality representation? 1: "Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. In Proceedings of Workshop at ICLR, 2013."
@statquest
7 ай бұрын
The number of values associated with each word is determined by the number of activation functions in the hidden layer. If you want 4 numbers, then add 4 activation functions.
Excuse me, what is your code editor?
@statquest
26 күн бұрын
I use jupyter.
Can you please create a playlist to demonstrate the coding part of Encoders, Decoders and Transformers? This is an earnest request from your FAN. Thanks.
@statquest
5 ай бұрын
I'm working on it right now.
do a video for ARIMA and VAR and their cousins please!
@statquest
7 ай бұрын
I'll keep that in mind.
🥳
@statquest
7 ай бұрын
YES! :)
BAMbastic
@statquest
7 ай бұрын
Thanks!
what is the different between pytorch and pytorch lightning sir ?
@statquest
7 ай бұрын
PyTorch Lightning is something that works with PyTorch that makes it easier to code and makes it easier to scale in the cloud and makes your code run faster in genernal.