This OpenAI Update is so awesome. translate paper about 3 pages per 1 request

Very helpful and thoughtful. I enjoy the way you present the topic and walk through!

Thank you for your insights. This is similar to the experiences I've had with longer context models.

Great examples - thank you.

Please keep up the good work and materials to ur channel! I love them so much!

I concure, I had the same experience with 16K model. Really helpful video. Thanks a lot.

Amazing video ! Thanks

You are smashing it, mate. I love your content! I am actually trying to create methodology with NLP to be able to evaluate transportation planning of municipalities in Germany. I believe there's huge potential. Your content is really helping me out here. Thanks a lot!

@samwitteveenai

Жыл бұрын

Thanks for the kind words. How do you find using all of these techniques in German? Any issues?

@grobetrotter5969

Жыл бұрын

@@samwitteveenai A generel issue with most of the LLMs is that they are trained with English data. For any other language than english OpenAI seems the way to go. I can't give you a lot of feedback so far. But I'll keep you updated whenever I know more.. ;)

@samwitteveenai

Жыл бұрын

Thats really interesting. I have seen something similar with a Chinese LLM that is bilingual. It would output English and then have Chinese just towards the end. Have you tried any of the open source models?

@grobetrotter5969

Жыл бұрын

​@@samwitteveenai I have used multilingual-e5-base from Hugging Face for zero-shot-classification and embedding. It seems to be working fine. I cannot really compare the results to other models yet. This is quite difficult because I have obtained a relatively large dataset by web scraping and none of the data is labeled. Furthermore I have tried the instructur-xl model for embeddings. Using the retriever and checking the output, it seemed reasonable. But it is very confusing when you read that the instructor model is not trained for multilanguage purposes. You said that you have used it with spanish texts. Would you rather go for a open source multilingual embedding model like multilingual-e5-base, a open source state-of-the-are model like instructor xl or go for the OpenAi ADA embedding which seems to be trained multilingually? As I have been given some financial resources from the uni I am going to use an OpenAI LLM. It's easy to use via the api and right now seems to be the best. That might change with orca but we'll see. Other than that I have not really been using open source models since I have very limited access to a GPU.

@henkhbit5748

Жыл бұрын

@@grobetrotter5969 Hi , I always thought that openAi ADA embedding is trained in english only. But if someone can confirm that it is multilingual, for example Dutch, than it would be great.

I think you have a point with the way they handle the token input on the 16k model. After some experiments with classifying a large json array and requesting strict format response it seemed to turn into nonsense/blanks after a while. Swapping back to the normal 3.5 its back to perfect classifications again

Nice job, Sam, as always. I played around with the 16k model. When I pass a giant 10k chunk to the model dilution kicks in. Whereas with small 1k chunks the model provides reliable responses to a prompt, with giant chunks the model misses some fine-grained relationships between prompt and the text. Wondering whether others experienced the same.

I doubt it is effective to refer to word counts, or any kind of concrete targets, to cause LLMs to adjust their output length. I would think you'd want to use words and phrases that are associated with the length of output you want. Such as "dissertation", "in-depth blog post", or "journal article."

@samwitteveenai

Жыл бұрын

Yes totally agree about word counts etc. Interesting ideas about "dissertation" etc. I will try those out.

Thanks for showing the new 16K chatgpt-turbo possibilties and (still) limitations.👍 How is the completion token calculated? I see that in the summarization example with a large response text it was 438 tokens and in the generation part it was 1248. Is the completion token calculation based on the response text?

Awesome video Can you make a gpt- engineer review

If the 16k input and output is still unreliable, maybe it would be better to create a simple function that can automatically break down down larger texts into 4k sections. It can then summarize those with ease. Finally get it to summarize the summaries. Once we master this, then we can really begin structuring data in new and exciting ways.

@samwitteveenai

Жыл бұрын

I wouldn't say it is unreliable, just that it doesn't work in the way a lot of people think it does.

Other experiments I've seen were only successful with full-context prompt trees whilst using the 8k GPT4. Not the others. I've had okay success with Anthropic but it performs best only expecting small outputs per pass, which is why they limit based on request and not tokens. So no, you cant use the 100k quite the way folk think either. You run out of requests quickly.

I'm curious. In the first case of summarization, why do you use LangChain but limit the input to 14k? Isn't it the point of LangChain to chunk the content in multiple requests so you can handle long content summarization. If you are going to limit the input to 14k, not use the OpenAI API directly without LangChain?

Thanks for sharing! In the "Generate the article" part of the notebook, should we pass in "questions" rather than "TOPIC_DESCRIPTION"? I tried fix this and get 3.5k token response using10 questions.

Facing the same issues as the open source guys. At least you can input a lot of text and have resonable answers on ooba I was running the 13b minotaur landmark model and the thing can go well over 2k even 3k but it writes so little. To get it more verbose requires reductions in repetition penalties and a turn up the temp at least from my limited experience :)

For some projects I work on we kind of need more than 16k tokens and then again a lot is zero shot under 2K context size,, It is a matter of perspective and need. These models at large context sizes are not as smart as you'd want them to be. Yet.

So, the bottom line is, the model's attention have gotten diluted amidst the 16k tokens and if following the current technology (transformers) the trend is expected to continue ?

You mentioned Singapore a few times. Do you live in Singapore?

@samwitteveenai

Жыл бұрын

Yes most of the time.

Hi sam , my name is shaziya i am following all longchine tutorial your videos it's very comprehensive. I have one question how can create autonomous sale agent with conversation agent and it has to follow my instructions what i have inform to my Agent and it's has to only response only to produce and not answering GK questions like where is USA . can you make video of autonomous sales agent like filip-michalsky/SalesGPT . This in githup kindly do the needful

Could you be being a bit hard on GPT on the story writing? What would happen if you took a more human like approach where we'd think of the topics then Google for ideas to flesh out the content for each idea. In the example you provide, the ideas and the story have to come from whatever set of weights were generated back in 2021 - limited as they may be as you suggest. It's good to know the summarization does work on long documents. Thanks.

@samwitteveenai

Жыл бұрын

I would love to see someone post a prompt that can get it to write 10k tokens in a single pass.

@christopherd.winnan8701

Жыл бұрын

@@samwitteveenai Norris McWhirter is patiently waiting to update the Guinness Book of AI Records.

might be the wrong place for it but are the Flan models even still worth talking about given all the development?

@christopherd.winnan8701

Жыл бұрын

Where can we learn more about Flan models? Do you have a TL:DR?

@samwitteveenai

Жыл бұрын

The Flan models are indeed cool but most only have a context window of 512 which makes them impractical for most work. There is so talk of training them more to extend that. Let's see. The 20B Flan model I did a video about has a context of 2k so it is better.