Since chatgpt blew up it's been tough to find technical content on chatgpt so thanks for pulling this up!

@lucasjackson7647

Жыл бұрын

Just chatgpt it lol

@technophobian2962

Жыл бұрын

One of the reasons for that is openAI not being very open.

@AkolytosCreations

Жыл бұрын

+1 to this. I have spent hours trying to find technical content like this. Videos either assume you know everything about AI and jump straight into in depth things (and even these videos are rare), or are so superficial it doesn’t really say anything. This was that perfect inbetween.

Very insightful. Following Dall-e, it seems OpenAI was a little bit more protective of their training IP (only a blog on ChatGPT - no paper). You have enough familiarity with the surrounding papers and tech to paint a clear picture of what their doing. Excellent work and again, very insightful!

@ariseffai

Жыл бұрын

Thanks DJ, appreciate the kind words :)

@laurenpinschannels

Жыл бұрын

by the way mutual information, I would love to see you make your subscription lists public

@b0nce

Жыл бұрын

For real, DJ, on every ML/DL/Math YT channel I like, I've seen your comment at least once :D

@Mutual_Information

Жыл бұрын

@@laurenpinschannels ha I didn't realize it was private. Switched! Enjoy :)

@danielhenderson7050

Жыл бұрын

Agreed, thank you for sharing

one of those elusive youtube gems. Wish there was more content out there for the serious nonexpert. Thanks!!

Thank you for making the video. Great balance of technical content and accessibility for people (like me) who aren't in the field.

A really fun video to watch, kudos to you for making such an esoteric topic easy to understand (at least in broad terms) for a layman as well.

Thankyou. I've been learning chatgpt to program microcontrollers and this video clear up a lot of questions and helps explain the common problems I get from the chatgpt bot output. I'm finding that it takes a lot of work on the part of the user to establish context, provide training examples, and to find the best wording to achieve your goal.

One of the only useful videos on ChatGPT on this platform. Great work

You are doing an amazing job explaining the complex concepts in a simple way. Keep up the good work!

Best video Ive watched describing ChatGPT! (and watched more than 20+) You have great insights!

Great work, Ari! Thank you very much for crafting the content, it's really easy to digest.

Amazing video. Thanks for publishing this. Going to dig through the rest of your videos too

That we need people on youtube that provide actual useful easy to comprehend knowledge, based on their leanring experience. Basicaly any human that have signigicant leanign expience and knowledge in one or more domains is a human chatgpt. Thanks for the content.

Very simple and effective explanation. Thank you.

Thank you so much for your efforts, this video was by far the most helpful for my project!

Brilliant. On aspect of Intelligence is a measure of one's ability to describe a complex topic into simplistic terms everyone can understand. My friend - you have that ability in spades. Congrats and Thank You !!!!!

Excellent video, thank you - definitely one of the best technical explanations of what is going on under the hood of ChatGPT I have found on YT to-date.

@TasteTheStory

Жыл бұрын

On my KZread channel, I tested how good ChatGPT is at writing movie scripts! I found the results to be interesting.

Technical, concrete and easy to follow explanation, good video 🔥

Very much valuable content explained with clarity I wonder why you channel haven't still exploded you earned a new sub and continue making videos on such topics

This is the best explanation of ChatGPT!

Best step-by-setp explanation !

Dude, your content is incredible!

Thank you so much! It is such a great video even for beginners!

good insight to how it works learned something new!

amazing, please make more!

Very well presented. Thanks!

Very well-made presentation, please make more! Subscribed

Great explanation and naration...! Thanks!

Cool video shot, well done, thanks for sharing :)

This is a very nice explanation, thanks! What tools do you use to make your videos?

@ariseffai

Жыл бұрын

Thanks! For this one I used a combination of keynote & FCP

Hey, just found your channel. Awesome stuff (currently studying for a masters in ML, it's crazy to see topics I've covered in class come up here)!

@billvvoods

Жыл бұрын

@Daniel Torres, Congratulations. Just curious but what was your bachelors in?

@DanielTorres-gd2uf

Жыл бұрын

@@billvvoods Mechanical Engineering!

@billvvoods

Жыл бұрын

@@DanielTorres-gd2uf very nice! I wish you the best in your studies. I’m now inspired 😉

@DanielTorres-gd2uf

Жыл бұрын

@@billvvoods Thanks, you as well! :)

Thank you! Very informative!

Nice explaination. Thanks

Hi Ari, really appreciate you made the video! It is great learning experience. Do you mind sharing the transcript on your website as well? For tech stuff, people like me learned better by reading than by watching videos. I tried use the extension to get the video script, but it is not 100% accurate so some tech words are not correct.

Well- explained video. So cool!

Thank you! Well explained.

This was so helpful thank you!!

Very good presentation!

Absolute gem ❤

On the same boat here, after minutes of going through click baits, finally a worthy explainer. Thank you.

the clearest ai expert on youtube

Great job. Going to show this to my class (Large Language Models for Lawyers, University of Houston Law Center)

Excellent review

Dang, this is a GOOD video. So many crap videos have been published on the topic. Hard to find one that has substance. THANK YOU!

@gpt-jcommentbot4759

Жыл бұрын

lol i hear this on every ai video.

this is so good, subscribed.

@yugen3968

Жыл бұрын

Scammer pos

Great content, Thanks

Great explanation.

Nicely done. Thanks for creating this video. Few quick questions/clarifications. (1) Given the reward model rates an entire response as opposed to each partially complete sentence as tokens are emitted, isn't the final stage also rating the reward for an entire possible sentence (that is terminated by a stop token?). Or do you believe the output sentence is rated for each token emitted until stop token? (2) Also was the use of the SFT also in the third stage for KL divergence calculation omitted in the figure because it was too much detail? (3) You mention 3000 words max limit. Is this an approximation for the max sequence length of 8k tokenized length (4) Lastly, do we know if the model parameters are 16bit floats or 32 bit? Thanks again for making this informative video.

@ariseffai

Жыл бұрын

Thanks! 1) Yes, the reward model rates an entire completed response. So an "action" here is a full response (sequence of tokens w/ ) emitted by the model. 2) Are you referring to the plot at 12:06? 3) The 3K words is an approximation to 4K max tokens, as described here: help.openai.com/en/articles/6787051 4) Great question! I'm not sure of the precision of the production model. Do post if you find it :)

@AR-iu7tf

Жыл бұрын

@@ariseffai thank you for your response. Regarding question 2- yes exactly. From the openai blog picture and the instructgpt paper I assumed three models were used in the final RL training - a copy of the SFT that became final production model(RL model ) with updated weights , the reward model and a frozen SFT model for the KL divergence computation that constraints the RL model to generate original sentence but not too far off from SFT. Is that your understanding too ? Regarding 3- certainly will post in case I find it . Thanks again !

Legend has returned - pls make more videos!

thank you great video, great detailed explanation

@ariseffai

Жыл бұрын

Thanks Nebras!

I'd love to know more about those "expert" conversations. Do you need to be an expert in the conversation matter or is it just used to make sure it's good at conversing (rather than getting the facts right)? How many of these expert conversations are useful? Is it a case of diminishing returns beyond a certain point? I'm guessing this isn't freely available information but it's fascinating to me.

Just wanted to thank you for these videos.

Nicely done. Thanks for creating this video. Few quick questions/clarifications. (1) Given the reward model rates an entire response as opposed to each partially complete sentence as tokens are emitted, isn't the final stage also rating the reward for an entire possible sentence (that is terminated by a stop token?). (2) Also was the use of the SFT also in the third stage for KL divergence calculation omitted in the figure because it seemed like too much detail? (3) You mention the upper limit is 3000 words. Is this an approximation for tokenized words that would a maximum sequence length of 8k? (4) Lastly, any idea if the parameters of the model is 16bit float or 32 bit float? Thanks in advance!

Great content!

Well done !

nice video, thanks !!!

Excellent insight dude! Awesome work. I need some help on time series algorithms? dataset with multiple parameters. can you help?

Great summary. I didn't follow when you said "we need to model to act during training" as a way of mitigating distributional shift... can you explain some more?

@ariseffai

Жыл бұрын

So basically, if the model takes zero actions during training, this means we'll have a big difference between the deployment distribution of states (when the model selects actions itself) and the training distribution (when the model merely observes the human's actions). There are different ways to have the model select actions during training. One is by using a standard reinforcement learning setup, as mentioned in the video. In that case, the policy model is directly rewarded for actions it itself executes. But another possibility comes from "on-policy" imitation learning, such as the DAgger algorithm. We iteratively execute the current policy to gather new training states, but then have an expert provide the correct action labels -- see arxiv.org/abs/1011.0686

This is a coherent nicely structured explanation of ChatGPT's architecture. Thank you for sharing this. BTW, how likely is it that OpenAI will create a new model with primarily supervised learning? I assume they are curating a new training set from both human responses and model-generated responses. It seems to me that a smallish self-supervised transformer model, trained in an autoregressive fashion from a well-curated knowledge base like Wikipedia and the Encyclopedia Britannica, etc., would be a great start for transfer learning from a curated supervised training set. Your video seemed to suggest this possibility. Moreover, it would be very interesting to run this side-by-side with a different architecture based on a vector database and semantic search for knowledge collection, retrieval, and context building. The results of this could be passed through an LLM for human readability and probabilistic generation. This should result in some sort of fuzzy-verified responses.

Very informative

Given that the scores used to train the reward function is small, compared to the universe of potential questions and answers, it's hard to see how a small training set can possibly be sufficient to train adequately. Still amazes me.

that was awesome

please make more videos like this

May I ask what technology you used to create such nice explanatory videos? Did you use 3b1b's manim engine? thanks.

@LSS94

Жыл бұрын

Very much looks like it!

@ariseffai

Жыл бұрын

Not for this one - just keynote and FCP. But I have used manim in a couple other videos :)

Thanks a lot for the explaination. How does it work during inference time to keep a conversation back and forth?Is the user's current chat session provided to the model as input along with a new user prompt?

@ariseffai

10 ай бұрын

That's right. There's a certain context window of previous text to which the model can attend (on the order of thousands of tokens). This will include both previous user inputs and model responses from the current conversation.

Love the explanation!! Also thanks for making the video darkmode 😊

@sjakievankooten

Жыл бұрын

@@TasteTheStory good videos mate, but no need to spam it here :)

@TasteTheStory

Жыл бұрын

@@sjakievankooten Not spaming just trying to connect with people who share the same interest. thanks for your note.

Nice Video

Can you please make a video on Midjourney as well?

Lmao this literally what i asked GPT today since I'm making a chatbot on Rasa. Looks like the algos are pointing me in the right direction for once!

13:12 The new bing (sydney) is able to link sources perfectly now

How does the reward model score a single action, when it is trained to choose between two actions? Or does the policy model actually generate k actions that the reward model can then score and then choose a reward knowing which action the policy model saw as the most probable one? I'd really appreciate an answer, thanks.

Is the operations from us as users part of the reward system ?

Very good video Can you maybe make an updated version now that chatgpt 4 is released and the new googel gemeni is about to come out for mixel input AIs

Hello, I want to work in this field. Now I'm a first year student studying informatics, how should I move towards it? Thank you!

Hi, can you make a small video to build ChatGPT with NLP based classification Model.

Hey, where could I approach you to clear a few things out about this...?

i used chatgpt to help we write a love letter and it went really well.

@brandonojalvo9775

Жыл бұрын

Your love is a lie

That's all very high level usage of neural networks. While some people think the basic foundations haven't set yet. Like for example 2 Siding ReLU.

Thanks so much for posting a clear explanation. After watching this, I feel like I do after I've been explained how a magic trick works: disappointed.

Are you saying that the 3.000 words can not be increased by just for example more ram usage per chat (chatgpt)?

What is the architecture of the policy model and how large is it? How does it use the pretrained LLM?

This was a wonderful explanation! Wouldn't it be expensive to have that much human capital evaluating and simulating chatbot responses? Seems especially so when you consider the wide amount of domains ChatGPT is able to provide reasonably correct responses to.

@CyberDork34

Жыл бұрын

Yes it is expensive. OpenAI outsources these tasks to countries like Kenya to save on these costs. It's kind of dubiously ethical but yeah

@roromaniac8

Жыл бұрын

@@CyberDork34 do you have a source that I could read about this? I haven’t been able to find something online.

Thanks!

@ariseffai

Жыл бұрын

Thanks Randall!

Cool bro

BEST!

What is the platform that OpenAI uses to build chatgpt. Like pytorch, tensorflow or something ?

It is possible to do most of this process with just the fine tuning api?

amazing video Ari. Where is the name from? Israeli?

Kerenn

good explanation!! i just wandering how / what chatgpt threshold for displaying no results? i have observe something like this for example : me : (example of 1 non professional gamer) chat gpt : i dont have enough data for him me : (example of 1 professional gamer in same game) chat gpt : *explain professional player* me :(asking the first player) chat gpt : *explain about that non professional gamer*

anyone know what the equation is at 4:08 , where i can find more on it?

I'm having trouble understanding supervised fine-tuning in this context. What are the labels? What is the task?

Thanks for the talk! You mention that the reward model is trained using cross-entropy loss as a binary classifier. I don't think that's accurate since you don't have a ground truth label for, say, response A (since the score is relative to others). The openAI paper just uses the negative log difference in scores between the higher and lower ranked response as the loss.

@ariseffai

10 ай бұрын

You're welcome! That's not quite correct. The classifier is trained to predict which of two responses is ranked higher by the human contractors. Then, the scalar logit output by the trained classifier for an individual response can be used as a reward signal.

the Math and some of the logic went over my head so I'm going to tray and summarise what I think I understood: ChatGPT is built on top it's predecessor InstructGPT which is mathematically trained with large data set to give spit to instructions given. However for ChatGPT, just spitting answers from instructions isn't enough and needs to be retrained over a method called 'Reinforcement Learning' which uses a 'reward model' to rank the next favourable answer. Did I get it? if not, then please tell me what I'm missing but in plain language because I know the above is flawed.

Codes as training data are only briefly mentioned?

Why does it care about the reward in reward reinforcement?

Plot twist : Content of this video was generated by chatGPT

PPO= operant conditioning?

According to ChatGPT it's memory is limited to only one prior message in the same conversation, beyond that it can't remember anything.

ChatGPT is like texting someone using only autosuggest, with most/all of the internet as the database. The real innovation in ChatGPT is compressing most/all of the public internet to just 350 GB, and an open source project has this down to 1.62 GB.

@D_Winds

Жыл бұрын

I would like to have this reservoir of data.

@uestcpandou

Жыл бұрын

Regard Chatgpt or other LLMs as database is a huge misunderstanding. Same as human brain, LLM is not so good at memorizing, compared to its ability of reasoning and fabricating new things. Combine traditional db and LLM to make them do what they are good at is the only way in the long run.

@nixedgaming

Жыл бұрын

Calling transformers with self attention and multiheaded attention “autosuggest” is wildly reductive and borderline disingenuous, even if it’s technically correct

@StephenGillie

Жыл бұрын

@@nixedgaming The best kind of correct! Others remind that calling this compression is also wildly reductive and borderline disingenuous, so at least I'm consistent.

@BooleanDisorder

Ай бұрын

Calling it autosuggest is like calling humans autosuggest. "Humans are just autosuggest with all of their lives as the database." People misunderstand what's what. You need to separate the task from what it actually is. The task is next word prediction. How it works is definitely not like autosuggest. The same way my task right now is next word "prediction" while writing this.

I'm learning about it's trained models and it's inner functions, just to create a perfect Jailbreak. thanks