HuggingFace Fundamentals with LLM's such as TInyLlama and Mistral 7B
Ғылым және технология
chris looks under the hood of huggingface models such as TinyLlama and Mistral 7-B. In the video Chris presents a high level reference model of large language models and uses this to show how tokenization and the AutoTokenizer module works from the HuggingFace transfomer library linking it back to the HuggingFace repository. In addition we look at the tokenizer config and Chris shows how Mistral and Llama-2 both use the same tokenizer and embeddings architecture (albeit different vocabularies). Finally Chris shows you how to look at the model configuration and model architecture of hugging face models.
As we start to build towards our own large language model, understanding these fundamentals are critical no matter whether you are a builder or consumer of AI.
Google Colab:
colab.research.google.com/dri...
Пікірлер: 46
The best video I've watched on KZread about LLM so far. You explain complex topics in an accessible language, clearly and understandably. You are doing a very good job. I'm eagerly waiting for the next videos :)
@ThomazMartinez
3 ай бұрын
same here
@chrishayuk
3 ай бұрын
Wow, thanks!, this one actually took a long time to get right, glad you liked it
Thank you so much Chris I truly believe if one has a great understanding of a subject he can teach it clearly and you simply did that ! God bless you
@chrishayuk
Ай бұрын
You are too kind, and thank you. Glad it was useful
Excellent explanation. Although I don't have a use case to fine-tune a model currently, I presume I will eventually it'll be great to have what you've shared in my back pocket. Thanks a bunch.
@chrishayuk
3 ай бұрын
Awesome, glad it was useful
Great video! Looking forward to your next videos…
@chrishayuk
3 ай бұрын
Yeah, next ones in series will be fun, glad you’re enjoying it
Great video. Just the right amount of detail. Thanks.
@chrishayuk
3 ай бұрын
Glad it was helpful!
Looking really forward to the next video.
@chrishayuk
3 ай бұрын
next one dropped
Amazing way to get people comfortable with the model architecture. Thank you so much for sharing your knowledge.
@chrishayuk
2 ай бұрын
Glad it was useful
Insanely valuable video. Thank you!
@chrishayuk
3 ай бұрын
Glad it’s useful
very well explained and useful
@chrishayuk
3 ай бұрын
So glad to hear that, thank you
Thanks being so so in details. That was really a refresher for me. Glad someone like you is doing such a good work.
@chrishayuk
3 ай бұрын
thank you, very much appreciate that
Thanks a lot Chris.
@chrishayuk
12 күн бұрын
glad it was useful
For the very first time, I finally get it, thanks to you. Thank you for your service to the community.
Thank you! This video brings light into the black box of LLM magic)
@chrishayuk
3 ай бұрын
more to come, the next set of videos reveal a bunch more
Excellent tutorial to get started with LLMs.
@chrishayuk
3 ай бұрын
Glad you liked it!
great video - many thanks!
@chrishayuk
2 ай бұрын
Glad you liked it!
Aways awesome content ❤
@chrishayuk
3 ай бұрын
Super glad it’s useful, thank you
How does the tokenizer decode sub-word embeddings? Specifically, how do you determine which sequence is concatenated into a word vs. standing on its own? As shown, the answer would be decoded with spaces between the embeddings, which wouldn't make "Lovelace" into a word.
@chrishayuk
3 ай бұрын
Certain tokens will have spaces others won’t so _lace would be a different token from lace. I have a deep dive of the tiktoken tokenizer where I spend a lot of time on this. I am planning to do a building a tokenizer vid soon as part of this series
@chrishayuk
3 ай бұрын
how the tokenizer for gpt-4 (tiktoken) works and why it can't reverse strings kzread.info/dash/bejne/gIGjqqqMh8mXfdI.html
@john6268
3 ай бұрын
@@chrishayuk Thanks, I'll check out the other video and looking forward to the next one.
great video. Thx!
@chrishayuk
3 ай бұрын
thank you
Great video.
@chrishayuk
3 ай бұрын
thank you
Great job sir, one video for me sir how to build llama APIs i want use my train own model now i want using in my website ..
@chrishayuk
3 ай бұрын
That’s where we are are working up to, but you can check out my existing fine tuning llama-2 video
Bro, just turn-on the Big Thank so I can donate you
@chrishayuk
3 ай бұрын
lol, not gonna happen but appreciate the gesture and glad you like the videos