(GPT-2) Language Models are Unsupervised Multitask Learners | Paper Explained
Ғылым және технология
Here’s another video from my GPT series where I analyze the GPT-2(Language Models are Unsupervised Multitasks Learners) paper. I took a closer look at data gathering process, results and safety concerns that prevented the initial public release of the model.
Paper:
d4mucfpksywv.cloudfront.net/b...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Links:
huggingface.co/datasets
openai.com/blog/better-langua...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Connect with me on:
Linkedin - / maciej-balawejder-rt8015
GitHub - github.com/maciejbalawejder
Medium - / maciejbalawejder
Buy Me a Coffee - [www.buymeacoffee.com/mbalawejder](www.buymeacoffee.com/mbalawejder)
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Timestamps:
0:00 Introduction
0:30 GPT-1 Recap
1:18 Abstract
2:20 Dataset
3:30 Byte Pair Encoding
6:30 Architecture
7:25 Results
7:50 Lambada
8:40 CBT
10:25 Winograd Schema Challenge
11:14 CoQA
11:35 Summarization
12:30 Translation
13:13 Question Answering
13:38 Conclusions
15:00 Safety Concerns
Пікірлер: 7
Thanks for the video man!
If you listen to his GPT1 presentation then this one will make more sense. Thanks Maciej, very well explained. :)
super helpful elaboration, thanks man, respect!
Superb explaination, please make similar explaination for gpt3 and bert
Thank you for explained Video ! I think the fact that open AI does not provide open source code is partly because of information security and technology competition .
Have you written a code for it just like gpt1?
@maciejbalawejder
Жыл бұрын
The architecture is the same as gpt-1, the only difference is configuration(number of layers, heads, and d_size)