The RAG chatbot you demonstrate is an excellent lesson with HuggingFaceEmbeddings. Regarding how to do it outside GPT being generic enough to have your own vectorDB on demand for any model I had wondered how that was done. Thanks for covering this really great stuff!

@juliensimonfr

5 ай бұрын

Glad it was helpful!

always making great and timely videos.

@juliensimonfr

8 ай бұрын

Glad you like them!

Thank you for your lectures.

@juliensimonfr

8 ай бұрын

You are very welcome

thanks julien, one can learn so much from these!

@juliensimonfr

8 ай бұрын

That's the idea 😀

Hey Julien, great job with the video. For QnA on corpus I'd recommend to generate hypothetical questions for each paragraph & ingesting them as well since those would have better similarity to the user input which is usually a question & can also help constrain the model to answer only closed domain questions.

@juliensimonfr

8 ай бұрын

Yes, that's a nice trick. I tried to keep things simple here ;)

Hey Julien, Thanks for an insightful talk last night at the AWS center!

@juliensimonfr

7 ай бұрын

You're welcome. Thanks for coming!

Thank you, gonna check it out tomorrow!

@juliensimonfr

2 ай бұрын

Have fun!

Thanks for this clear explanation.

@juliensimonfr

2 ай бұрын

Glad it was helpful!

Thanks for the video!

@juliensimonfr

Ай бұрын

You're welcome!

Thanks a lot! It was very, very helpful.

@juliensimonfr

5 ай бұрын

You're welcome.

Hi Julien, thanks for your video, pretty clear explained ;-)

@juliensimonfr

5 ай бұрын

Glad it was helpful!

Hi Julien, Thank you for this video. It's helping me learn a lot. I was trying to run the code. When I attempt the zero shot example, my output is quite different from whats shown in the video. I tried to split it, but I get something like this - [answers: * 1) The trend is to invest more in solar energy in China. * 2) The trend is to invest less in solar energy in China. * 3) The trend is to invest the same amount of money in solar energy in China. * 4) The trend is to invest more in solar energy in the United States. * 5) The trend is to invest less in solar energy in the United States. ] Can you please explain why this is happening and how it can be fixed?

Sagemaker with langchain streaming option is generating output

Thanks Julien! very nice video. very curious if there are some compare between bge-small with ada-002 when used in RAG.

@juliensimonfr

7 ай бұрын

Hi, please check our embeddings leaderboard at huggingface.co/spaces/mteb/leaderboard. ada-002 is #15, bge-small is #8 :)

It throws : KeyError: 'Blocks' after running the cell with boto3.client('textrac') thrown by the loader.load(), from parser in langchain

Superr video, Thanks for trying using open source solutions...

@juliensimonfr

8 ай бұрын

Glad you liked it

Thanks Julien, for the good tutorial! Some use pinecone, do you see differences/advantages of using faiss over pinecone? Thank you

@juliensimonfr

5 ай бұрын

FAISS is a simple lightweight open source solution. Pinecone is a fully managed, closed source DB running in the cloud. Depends what you're looking for, and how much work you want to do on managing the solution :)

thanks it was really informative, can do demonstrate fine tuning LLM's with lora and Qlora? In your experience, RAG has better performer over fine tuning ?

@juliensimonfr

Ай бұрын

Llama2 fine-tuning with Qlora: kzread.info/dash/bejne/jJmqmKhphJSyY7w.html. IMHO RAG and fine-tuning solve different problems and are complementary. RAG lets you access fresh company data and gives you some domain adaptation. Fine-tuning gives you better domain adaptation and lets you customize guardrails and tone of voice.

Thanks!

@juliensimonfr

3 ай бұрын

You bet!

Thanks for the tutorial.In my case,i can't use Mistral somehow due to some restrictions on AWS test account.I have used FLAN-T5 but it is giving this error.ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (422) from primary with message "Failed to deserialize the JSON body into the target type: missing field `inputs` at line 1 column 503".

@juliensimonfr

5 ай бұрын

The input format for T5 is quite different, so sending a Mistral-formatted message won't work. Not sure what restriction you're facing, but maybe TinyLlama would work? I think you would only have to adapt the prompting format in the content handler.

Thank you Julien - this is super useful and comes at the right time during my writing season (you know what I'm talking about :-) ) As someone else mentioned in the comment, I also received an error when calling Textract. I solved it by adding `pip install amazon-textract-textractor -qU` - hope it might help others

@juliensimonfr

7 ай бұрын

Ok, good to know. Thanks Seb and good luck with the writing ;)

@SebastienStormacq

7 ай бұрын

also 'pip install pip install faiss-cpu' :-)

godlike

I want to embed large data. In this case, if I want to embed document without a GPU notebook ml.t3.medium, is it possible to deploy the embedding model as well in some ml.g5.large GPU instance to make the processing faster?

@juliensimonfr

3 ай бұрын

Sure, it's what you would do for production.

Great video, But what if user question is related to chat history and it may contain short cuts like he/she/that/it etc then how to handle such cases

@juliensimonfr

5 ай бұрын

Langchain has different ways to handle this, e.g. python.langchain.com/docs/modules/memory/types/buffer

@Thirumalesh100

5 ай бұрын

Thanks@@juliensimonfr , Basically it is question rephrase request by passing entire chart history, tried this approach which has cost and token limit problem Looking for other alternative for the same

can you tell how to get the key for sagemaker to work here?

@juliensimonfr

2 ай бұрын

Not sure what you mean. Are you looking for a SageMaker tutorial ? See docs.aws.amazon.com/sagemaker/latest/dg/gs.html

Thx for the video :) can you update your vector database by a few lines ( if you want to add data to your knowledge base) automatically by running a python script or something like that?

@juliensimonfr

7 ай бұрын

Sure, you can keep adding embeddings anytime you want.

@da-bb2up

7 ай бұрын

oh thats nice :) thx for the answer@@juliensimonfr

Hi Julien. The code is not working when I try to run it. I think the error I am getting is related to Sagemaker credentials. I made an account just now but don't know where to get information where I can plug into your code to make this work.

@juliensimonfr

8 ай бұрын

Start here: docs.aws.amazon.com/sagemaker/latest/dg/howitworks-create-ws.html. Create a notebook instance and make sure its IAM role includes the SageMakerFullAccess and TexttractFullAccess managed policies. Once you've done that, the notebook will run as is.

@kevinngo3722

8 ай бұрын

Thanks for your reply! It seems that this leads me to make a Jupyter notebook. How do I integrate this to do what you're showing on Colab in the tutorial?@@juliensimonfr

how can I call onto my react frontend?

@juliensimonfr

2 ай бұрын

A SageMaker endpoint is an HTTPS API, so you can plug it in anything. You should be able to find lots of examples out there.

Y r u deploying first in sage maker

@juliensimonfr

5 ай бұрын

Because I don't want to manage any infrastructure :)

Seems like a great video, but I can't move from the starting line. You seem to be skipping over very important details about how to deal with the HuggingFace tokens, AWS security keys, regional compatibility settings with Sagemaker, etc. For example, when running the copied SageMaker code, I get "ValueError: Must setup local AWS configuration with a region supported by SageMaker", but no region seems I try seems to work. Did you cut all the authentication code from your demo? Obviously you don't want to disclose security keys, but at least show/explain that part of the setup code and simply redact the sensitive information.

@juliensimonfr

14 күн бұрын

How about going through Hugging Face 101 and SageMaker 101 first?

@Azazello1482

13 күн бұрын

@@juliensimonfr Yes, clearly I'll need to do this! Nonetheless, as an educator myself, I think my point is still useful. It's helpful to learners to mention parts that you skip over. You don't have to teach it in this video, but it would be helpful to mention that there are steps one must perform that are not shown in this video.