Word Embedding in PyTorch + Lightning

Word embedding is the first step in lots of neural networks, including Transformers (like ChatGPT) and other state of the art models. Here we learn how to code a stand alone word embedding network from scratch and with nn.Linear. We then learn how to load and use pre-trained word embedding values with nn.Embedding.
NOTE: This StatQuest assumes that you are already familiar with Word Embedding, if not, check out the 'Quest: • Word Embedding and Wor...
If you'd like to support StatQuest, please consider...
Patreon: / statquest
...or...
KZread Membership: / @statquest
...buying my book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
statquest.org/statquest-store/
...or just donating to StatQuest!
paypal: www.paypal.me/statquest
venmo: @JoshStarmer
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
1:53 Importing modules
2:48 Encoding the training data
6:55 Word Embedding from scratch
16:54 Graphing the embedding values
21:17 Printing out predicted words
20:37 Word Embedding with nn.Linear
28:12 Loading and using pre-trained Embedding values with nn.Embedding
#StatQuest #neuralnetworks #transformers

Пікірлер: 75

  • @statquest
    @statquest7 ай бұрын

    Here's the code: lightning.ai/lightning-ai/studios/statquest-word-embedding-with-pytorch-lightning?view=public§ion=all To learn more about Lightning: lightning.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

  • @loserc1854

    @loserc1854

    2 ай бұрын

    waiting for ur next book

  • @drintro
    @drintro4 ай бұрын

    The best part of this example is the way the first example is written with single dimension arrays for all of the parameter values. That makes the matrix operations explicit and more clear even to an experienced developer. I recommend writing and debugging the code from watching the video. There are small things that you might get wrong that will teach you something.

  • @statquest

    @statquest

    4 ай бұрын

    Thank you! I'm glad you liked the example.

  • @scottp131
    @scottp1317 ай бұрын

    I bought The StatQuest Illustrated Guide to Machine Learning, and it's absolutely amazing, same with every single one of your videos! I highly recommend that book to anybody who is interested in learning about this stuff! Thank you thank you thank you so much for taking the time to put together the content you make, I can't believe how well you paint this stuff into perspective! You're a freaking awesome person Josh, I'm still absolutely elated that I stumbled across this channel! I never would have expected to be so consumed with learning this and everything about this!

  • @statquest

    @statquest

    7 ай бұрын

    Thank you very much! I'm really glad you enjoy my videos and my book! Thank you! :)

  • @vishnumuralidharan9858
    @vishnumuralidharan98584 ай бұрын

    Hi Josh, I just want to say that you are an absolutel godsend to the ML and DS community. I have been following your content since 2020 and you have never let me down. I hit a block with PyTorch code implementation and your videos simply untangled a lot of my mental knots. Keep up the great work!

  • @statquest

    @statquest

    3 ай бұрын

    Awesome! Thank you!

  • @exxzxxe
    @exxzxxe4 ай бұрын

    Josh- you are a Master in making the difficult comprehensible!

  • @statquest

    @statquest

    4 ай бұрын

    Wow, thanks!

  • @tom19860526
    @tom198605264 ай бұрын

    I like your videos. Both the slides and explanations are very detailed and clear. I cherish them every time I watch them. It is a good learning video. thank you very much.

  • @statquest

    @statquest

    4 ай бұрын

    Thank you very much! :)

  • @InglesConConfianza
    @InglesConConfianza6 ай бұрын

    Just finished the Deep Learning playlist. Thank you so much for this great work.

  • @statquest

    @statquest

    6 ай бұрын

    BAM! Thank you very much! You deserve an award!

  • @itaydagan7459
    @itaydagan74596 ай бұрын

    Josh you are one of a kind!! Thank you for all the content!

  • @statquest

    @statquest

    6 ай бұрын

    Thank you!

  • @eliyahubasa9401
    @eliyahubasa94017 ай бұрын

    Great video, a great way to study. Thank you.

  • @statquest

    @statquest

    7 ай бұрын

    Thanks!

  • @sweetlearning6629
    @sweetlearning66297 ай бұрын

    Thing is; i always look forward to new videos from you, cause i know it would be awesome as always. I would want to see more videos on computer vision tho, just hope you'd consider this. Thanks

  • @statquest

    @statquest

    7 ай бұрын

    I'll keep that in mind.

  • @itsawonderfullife4802
    @itsawonderfullife48027 ай бұрын

    I too, love your work and specially appreciate your playful style and all the movie references. ;)) Thanks for your videos, all of them.

  • @statquest

    @statquest

    7 ай бұрын

    Thank you very much!

  • @RaynerGS
    @RaynerGS7 ай бұрын

    You rock. I love your work. Salute from Brazil.

  • @statquest

    @statquest

    7 ай бұрын

    Muito obrigado! :)

  • @nourinsiddiqueananna
    @nourinsiddiqueananna7 ай бұрын

    Yayy !! I was wanting a video like this !!!

  • @statquest

    @statquest

    7 ай бұрын

    BAM! :)

  • @diegoandradex12
    @diegoandradex127 ай бұрын

    Great video as always

  • @statquest

    @statquest

    7 ай бұрын

    Thanks again!

  • @d25102
    @d251027 ай бұрын

    Thank you!

  • @statquest

    @statquest

    7 ай бұрын

    :)

  • @marwolaeth111
    @marwolaeth1116 ай бұрын

    StatQuest is so inspiring! Please consider making a 'Quest about Geometric deep learning. What do you think?

  • @statquest

    @statquest

    6 ай бұрын

    I'll keep that in mind.

  • @abhilashb722
    @abhilashb7226 ай бұрын

    This is really great 🎉🎉🎉. Can you please continue this pytorch + lightning series

  • @statquest

    @statquest

    6 ай бұрын

    Yep! Working on it right now.

  • @kisholoymukherjee

    @kisholoymukherjee

    6 ай бұрын

    And hope you will also add them to the Deep Learning Playlist. Keep rocking (and BAM-ing!) @@statquest

  • @statquest

    @statquest

    6 ай бұрын

    @@kisholoymukherjee Will do!

  • @TJ-hs1qm
    @TJ-hs1qm7 ай бұрын

    StatQuest Time!!

  • @statquest

    @statquest

    7 ай бұрын

    BAM! :)

  • @kartikchaturvedi7868
    @kartikchaturvedi78687 ай бұрын

    Superrrb Awesome Fantastic video

  • @statquest

    @statquest

    7 ай бұрын

    Thanks!

  • @elodiebeitman8251
    @elodiebeitman82516 ай бұрын

    Hi! I also bought The StatQuest Illustrated Guide to Machine Learning - It's awesome! Triple BAM! I think there might be a small error with the formula on page 135 (Naive Bayes: FAQ Part 3) though, FYI! Thanks again!

  • @statquest

    @statquest

    6 ай бұрын

    This is a known error in the book. All of them are listed here: statquest.org/sigml-errata/

  • @yashsonune4391
    @yashsonune43916 ай бұрын

    Thank you again for the quality content. 🔥. Btw is there any plans for teaching about fine-tuning Language models. I heard this paper (Universal Language Model Fine-tuning for Text Classification) is of higher relevance and backbone for many state of the art solutions.

  • @statquest

    @statquest

    6 ай бұрын

    I'm working on one.

  • @user-xl9lj2qo4r
    @user-xl9lj2qo4r5 ай бұрын

    Hi josh Starmer, after learning ML, whether this series is enough for a s beginner to learn deep learning? and do any additional things to know other than this series for deep learning?

  • @statquest

    @statquest

    5 ай бұрын

    It's a great way to start.

  • @gsestream
    @gsestream6 ай бұрын

    as a related thing, finding closest things on a map/grid, without making a cell division system, sort all dimensions then check if they collide, like spheres or aabb bounding boxes, n log n time complexity, as opposed to n^2 time complexity, similar to grid cell partitioning of data, for clustering, or k-nearest clustering, but just finds closest at range, any number, very fast. should be applicable directly in high dimensionality complex data analysis, dimensionality reduction, covariance matrices, yep its designed to be fast

  • @vigneshvicky6720
    @vigneshvicky67207 ай бұрын

    Plz start teaching yolov8 which is used for object detection, segmentation, classification ... Every problem will be solved plz plz

  • @statquest

    @statquest

    7 ай бұрын

    I'll definitely keep that in mind! Keep reminding me on future "in PyTorch + Lightning" videos.

  • @vigneshvicky6720

    @vigneshvicky6720

    7 ай бұрын

    @@statquest sure sir but try to do it from scratch because everyone are using build in one so many of them dont know what is going inside. I would like to learn it from scratch so that I can finetune architecture myself . Do it as soon as possible🙏

  • @lebronjames8507
    @lebronjames85077 ай бұрын

    Do u guys have an education I can get somewhere? I saw u guys had a probability basics playlist but do u have some type of course to take me through every single subtopic of stats?!

  • @statquest

    @statquest

    7 ай бұрын

    Not yet.

  • @d_b_
    @d_b_7 ай бұрын

    👍So, these are word embeddings. Do you know/think that sentence embeddings and text embeddings differ much from this process?

  • @statquest

    @statquest

    7 ай бұрын

    I'm pretty sure they are the same, but I'm not certain.

  • @RaynerGS
    @RaynerGS7 ай бұрын

    The original work [1] uses a multidimensional vector for each word instead of a unique real value. For instance, Troll2 = [,025, 0,735, 0,256, 0,145], in this case, four dimensionality. In the paper, the authors use a matrix instead of a neuron. Question: using neurons, how would you increase the word dimensionality representation? 1: "Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. In Proceedings of Workshop at ICLR, 2013."

  • @statquest

    @statquest

    7 ай бұрын

    The number of values associated with each word is determined by the number of activation functions in the hidden layer. If you want 4 numbers, then add 4 activation functions.

  • @Johan-zs9xh
    @Johan-zs9xh26 күн бұрын

    Excuse me, what is your code editor?

  • @statquest

    @statquest

    26 күн бұрын

    I use jupyter.

  • @debarunkumer2019
    @debarunkumer20195 ай бұрын

    Can you please create a playlist to demonstrate the coding part of Encoders, Decoders and Transformers? This is an earnest request from your FAN. Thanks.

  • @statquest

    @statquest

    5 ай бұрын

    I'm working on it right now.

  • @iurgnail
    @iurgnail7 ай бұрын

    do a video for ARIMA and VAR and their cousins please!

  • @statquest

    @statquest

    7 ай бұрын

    I'll keep that in mind.

  • @mikinyaa
    @mikinyaa7 ай бұрын

    🥳

  • @statquest

    @statquest

    7 ай бұрын

    YES! :)

  • @arseniykan
    @arseniykan7 ай бұрын

    BAMbastic

  • @statquest

    @statquest

    7 ай бұрын

    Thanks!

  • @wibulord926
    @wibulord9267 ай бұрын

    what is the different between pytorch and pytorch lightning sir ?

  • @statquest

    @statquest

    7 ай бұрын

    PyTorch Lightning is something that works with PyTorch that makes it easier to code and makes it easier to scale in the cloud and makes your code run faster in genernal.