Upcoming Events
Join our Meetup group for more events!
www.meetup.com/data-umbrella
[53a] Sebastian Raschka: Introduction to PyTorch
Video 1/3: • [53a] Intro to PyTorch...
[53b] Adrian Wälchli: Scaling Up with LightningLite
Video 2/3: • [53b] Scaling Up with ...
[53c] Q&A: Sebastian Raschka & Adrian Wälchli (PyTorch, LightningLite)
Video 3/3: • [53c] Q&A: Sebastian R...
Key Links
- Transcript: github.com/data-umbrella/even...
- Meetup Event: www.meetup.com/data-umbrella/...
Resources
- DevCon 2022: www.pytorchlightning.ai/event...
- Sebastian's slides: github.com/PyTorchLightning/d...
- Adrian's slides: github.com/data-umbrella/even...
- 30% book discount: www.amazon.com/gp/mpc/A2MFJW5...
- Suggested resource: github.com/rasbt/deeplearning...
- github.com/rasbt/deeplearning...
- Sebastian's book: sebastianraschka.com/books/#m...
- kfold: github.com/rasbt/deeplearning...
- model inspection: pytorch.org/tutorials/interme...
- Link to code examples: github.com/PyTorchLightning/d...
Community Announcements
- Adding timestamps: github.com/data-umbrella/even...
[53a] Video 1: Intro to PyTorch
Agenda
00:00 Reshama introduces Data Umbrella
08:25 Sebastian begins
09:50 What is PyTorch? (tensor library, automatic differentiation engine, deep learning library)
10:50 TensorFlow vs PyTorch: why PyTorch is so popular
12:55 PyTorch: tensor library (rank-x tensor: scalar, vector, matrix, 3D tensor, 4D tensor
15:40 tensor library, torch.tensor ~= numpy.array)
17:19 PyTorch: automatic differentiation support
25:28 automatic differentiation in PyTorch
26:30 autograd
28:12 PyTorch: deep learning library
28:30 3 Steps in Neural Network Training
29:39 Defining the Model
33:50 Step 1: Define forward method
37:28 Step 2: Defining the training loop (initialize the model and optimizer)
41:10 Iterating over the training examples
42:52 Computing the predictions
45:10 Computing the backward pass (backpropagation)
optimizer.zero_grad() (remember to reset the gradients for each iteration)
46:41 Updating the model weights
47:14 Tracking the performance
47:52 no_grad() (we don't care about gradients here, we don't need to construct the computation graph)
48:52 Why do I like PyTorch?
50:38 Live Demo
51:10 Developer conference: Lightning DevCon
52:26 demo in Jupyter Notebook
[53b] Video 2: Scaling Up with LightningLite
[53c] Video 3: Q&A with Sebastian and Adrian
Event
This talk will introduce attendees to using PyTorch for deep learning. We will start by covering PyTorch from the ground up and learn how it can be both powerful and convenient. At times, Machine learning models can become so large that they can't be trained on a notebook anymore. Being able to take advantage AI-optimized accelerators such as GPU or TPU and scaling the training of models to hundreds of these devices is essential to the researcher and data scientist.
However, adding support for one or several of these in the source code can be complex, time consuming and error-prone. What starts as a fun research project ends up being an engineering problem with hard to debug code. This talk will introduce LightningLite, an open source library that removes this burden completely. You will learn how you can accelerate your PyTorch training script in just under ten lines of code to take advantage of multi-GPU, TPU, multi-node, mixed-precision training and more.
About the Speaker: Sebastian Raschka
Sebastian is a machine learning and AI researcher with a strong passion for education. As Lead AI Educator at Grid.ai, he is excited about making AI & deep learning more accessible and teaching people how to utilize AI & deep learning at scale. Sebastian is also an Assistant Professor of Statistics at the University of Wisconsin-Madison and the author of the Machine Learning with PyTorch and Scikit-Learn book.
- LinkedIn: / sebastianraschka
- Twitter: / rasbt
- GitHub: github.com/rasbt
About the Speaker: Adrian Waelchli
Adrian is a research engineer at Grid.ai developing and maintaining PyTorch Lightning, a library for researchers and deep learning practitioners built on top of PyTorch, minus the boilerplate.
- LinkedIn: / adrian-waelchli
- GitHub: github.com/awaelchli
- Twitter: / adrianwaelchli
#pytorch #python #deeplearning