Every Friday lunchtime, the whole of endjin get together for a "Show & Tell" session. These are essentially lightning talks on any subject; they can based on what someone has been working on that week, it can be a topic someone has been researching, it can be a useful tip or trick, it can be technical or non-technical; it just has to be something the speaker has found interesting, and wants to share with the whole company.
This week Ian Griffiths, Technical Fellow, gave a brief overview of investigations into using TorchSharp to model Tensors, to use backpropagation to train Neural Networks, in order to understand the principals behind building your own Generative Pre-Trained Transformer (GPT).
Contents:
- 0:00 What is Torch / PyTorch / TorchSharp?
- 0:45 Representing numbers as Tensors
- 2:21 Tensors and the .NET Type System
- 4:07 Multiplying Arrays
- 5:16 Tensors and Neural Networks
- 6:23 GPUs and CUDA
- 8:35 How inputs affect output
- 12:21 What's backpropagation?
- 21:33 Building a GPT from scratch
- 24:07 Which libraries were used?