Anyone comes across this in 2024 (jan ), the command switches with new autotrain version is autotrain llm --train --project-name josh-ops --model mistralai/Mistral-7B-Instruct-v0.2 --data-path . --use-peft --quantization int4 --lr 2e-4 --train-batch-size 12 --epochs 3 --trainer sft . Great, Video, thanks Abhishek

@BrusnickiRoberto

5 ай бұрын

After finetuning it, how to run it?

@vinodb4339

17 күн бұрын

​@@BrusnickiRobertoHey hi did you run it??

@BrusnickiRoberto

17 күн бұрын

@@vinodb4339 no

That's Awesome, nothing better than this way of training large language model. Super easy ❤

Thank you for the on-point tutorial.

Appreciate it, and request to continue making such videos🎉

Keeping it this simple is something very few people are able to do. Very well explained. This can be understood by even a beginner. Atleast the execution if not the intuition behind it. Kudos

Incredibly helpful video, I appreciate that you took the time to create this! Great stuff

Thank you Abhishek! This is phenomenal.

Excellent, thank you so much. I will try.

Astonishing content Man 🔥🔥🔥 🚀

Amazing, tutorials at light speed! Llama 2 was just released! 😮

@abhishekkrthakur

11 ай бұрын

🙏🏽

@AIOdysseyhub

11 ай бұрын

😂😂Yeah exactly

@weebprogrammer2979

11 ай бұрын

This man is a genius lol

Thanks for sharing!

thanks for sharing!

Great Job :)

Nice job!

Great video! Would be great if in some future vid you could go into depth on the training hyperparameters and perhaps also talk about what size your custom datasets should be.

@abhishekkrthakur

11 ай бұрын

sometimes I do that. however, this model would have taken wayy too long to train. im training a model as i type here and if i get good results ill share both model and params 🙂

@emrahe468

11 ай бұрын

@@abhishekkrthakur guess no good luck with the training :(

Thanks for the tutorial! A couple questions for you. Is there an approach you're using to test quality and verity that the training data has influenced the weights in the model sufficiently to learn the new task? And second, can you use the same approach for unstructured training data such as using a large corpus of private data to do domain adaptation?

Informative video, hopefully one day I will get a task that requires me to finetune a LLM

@abhishekkrthakur

11 ай бұрын

or you can just do it for fun 🤗

Thank you brother

Great tutorial! Can you also put up one video teaching on how to merge the fine tuned weights to the base model and do inference? Would like to see an end-to-end course. Thank you!

@adamocheri3513

10 ай бұрын

+1 on this question !!!!

@devyanshrastogi

8 ай бұрын

any updates guys?? I really want to know how to merge the fine tuned model with the base model and do the inference. Do let me you have any resources or insights about the same

@kopamed5024

5 ай бұрын

@@devyanshrastogi also need this answered. have you guys had any success?

thank you very much

Hi Abh Your method is inspiring and commendable. How do we read the csv or json training dataset we prepared instead of the hugging face dataset you used?

Nice video Abhishek! I am curious to know about custom data for LLMs. What is the ideal (good quality) data size (e.g., #rows), to fine-tune these models for good performance, not necessarily it should be big data of course. Thanks!

Thanks!

thanks bro ❤

Hi Abhishek, is the auto train using LORA or prompt tuning as the PEFT technique?

what kind of GPUs do you have? how big was your dataset and how long did it take to train? what is the smallest fine-tuning data set size that would be reasonable?

There is only one thing I want to see. I want to see you using the final result and prove it actually works. Thank you.

For fine-tuning of the large language models (llama-2-13b-chat), what should be the format(.text/.json/.csv) and structure (like should be an excel or docs file or prompt and response or instruction and output) of the training dataset? And also how to prepare or organise the tabular dataset for training purpose?

Thanks for this great explanation. Can you provide the link to you training data?

Great Video. i love it and can't wait to try it. Now that Llama2 is out... is it better to FineTune a model or try to create your own Model?

How do you set the training data? I see different people using different formats? Does it matter or is the only requirement that it has to be structured meaniningfully?

First thank you for the great tutorial , I have one question : I am trying to finetune the model on Japanese , do you have any advice for that ? I have tried the same script as you did but it didn't work; it produced some gibberish after the training finished , I am guessing it is a tokenizer problem, what do you think ?

Nice vid Should we label input tokens to -100 to focus the training on the prediction? I see no one doing it

Thanks Kaggle grand master :) you got an channel.

Thanks for the comprehensive tutorial. Can this be done using chat logs to build a clone of your friend? I have done this with GPT3.5 finetuning using prompt->response. The prompts are questions generated by ChatGPT based on the chat log message. Can the same thing be done with Instruction->Input->Response? Thank you very much man.

I know it could be too much but could you also make a video of how to prepare the data? I have like 'question' and 'answer' but I am strugging to make it to a trainable data set into that kind of csv so I could use it!

Superb video! Does autotrain allow to export finetuned model as GGML file? Or can it be used with GGML file?

The way Abhishek side eyes before stopping the video and resuming is is soo crazy 🤣🤣😅

@abhishekkrthakur

11 ай бұрын

lol. big screen. button too far 🤣

Thank you for the video. I am new to this, so I am trying to set it up on my windows PC. When I am trying to install the latest version of autotrain-advanced==0.6.2, I get an error saying: trition==2.0.0.post1 cannot be found. Which I believe is only available on Linux. So is it possible to use autotrain-advanced on windows?

I trained model using autotrain in same way as you suggested and model file is stored. Now I need to use this model for prediction. Can you shed some light on this as well?

How do you convert it to work with Ollama? I setup the model file and it doesnt seem to know anything from my training.

Hi Abhishek, Much appreciated. How would things change if we were to use simple fine tuning? That is, just a large single code file to learn from, to tune code-llama, phi2, etc..

When will I be able to see the model that I have trained? Once all the training is complete?

Can you make a video for fine tuning in silicon macs ?

How is your ipython giving suggestions? I want the same set up. Please make a video on these types of set up that assists in coding and other processes.

I just bought a RTX 4090 Founders Edition. Could you tell on a particular example were I could run into limits with card when training LLMs locally? I personally think that I'm safe for the next few years and I will not run in any problems.

Hey, could you guide us how to train custom dataset on llama2? How to prepare our dataset for training?

As always very useful and short without wasting anyone's time. Thank you. Just I'm a bit confused about the prompt formatting you have used here - "### Instruction: ### Input:... etc" while Llama official is "[INST] {{ system_prompt }}{{ user_message }} [/INST]" and on TheBloke's page it says "SYSTEM: {system_prompt} USER: {prompt} ASSISTANT:"

@ahmetekizx

8 ай бұрын

I think this isn't mandatory, it is a suggestion.

Thank you for this interesting video. How could we change the encoding to ''latin-1' in order to train on french language ? thank you.

Hi I have a question , it´s posible to use "autotrain" without cuda, with rocm support of AMD GPU ?

Hi when we tried fine tuning MPT LLM using autotrain, getting the error ValueError: MPTForCausalLM does not support gradient checkpointing. Any help you can offer on this pleas?

hello, thank you. i really need to know: does this pad appropriately? also, how does it internally split it into prompt completion? Can i make up roles like ### System? does it complete only the last message?

How do you deal with response in the dataset that has newline characters?

Do you have to use [INST]...[/INST] for indicating the instructions? I think the original Llama 2 model was trained with these tags, so I am a bit puzzled if you have to use the tags in the csv or they are added internally ?!

@abhishekkrthakur

11 ай бұрын

in this video, im finetuning the base model. you can finetune it anyway you want. you can even take the chat model and finetune it this way. if you are using a different format for finetuning, you must use the same format while inference in order to get the best results.

Thanks for the video. i have a doubt that how can i make a package of it so others can also use it offline by just installing the application

Great tutorial! Thank you! Maybe I missed it but, at which point do you enter your HF token?

@abhishekkrthakur

11 ай бұрын

You dont. You login using "huggingface-cli login" command. There's also a similar command for notebooks and colab. :)

If there are a multiple input values how that have to be inserted in the cvs data ?

Can we do this on a CPU? I am using a GGmL model.. please share your thoughts

Hello @abishekkrthakur can I train summarization models with autotrain advanced?

Thank you ,what is diff between instruction and input

Beautiful content, I have a side question, what tool you are using to have "copilot"-like suggestion in your terminal? Thx again for the video

@jessem2176

11 ай бұрын

I use Hugginfaces co pilot. - it works pretty well and super easy to set up and free..

@ahmetekizx

8 ай бұрын

@@jessem2176 Thanks for the recommendation, but did you mean HuggingFace Personal-copilot Blog?

How can i load the model for inference if i stop training after 1 epoch out of 3 epochs.

How to shard as you mentioned towards the end?

Where can i find to code you worte and your dataset? I would like to use it as an exemple for testing

I have llm downloaded can I fine tune it with downloading from huggingface.

In my experiment, it not create the [config.json] what am I doing wrong?

Thanks for this great video, but how to fine tune or train for question answer data set

Hey Abhishek, Thanks for sharing this amazing tutorial. Can I do this on my RTX 3070Ti 8GB GPU? If yes, what batch size would be preferable?

@abhishekkrthakur

11 ай бұрын

8GB sounds a bit low for this. maybe try bs=1 or 2? but tbh, im not sure if it will work. Might work fine for a smaller model!

I am running out of memory when testing it on the free-version of google colab, did you use the exact same tuning parameters as described in the video?

@abhishekkrthakur

11 ай бұрын

yes. you can reduce batch size. note, you need to use different model path if you are on colab or it will run out of memory. see description for more details

Is Gpu Machine necessary for llama 7b to be trained?

Thanks for the tutorial. On a Linux 5.15.0-71-generic #78-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux machine, I get following error when training llm with the small dataset. File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2819, in from_pretrained raise ValueError( ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modules in 32-bit, you need to set `load_in_8bit_fp32_cpu_offload=True` and pass a custom `device_map` to `from_pretrained`. What could be the problem? Is it possible to share the data.csv that you have with single row that I can take as reference to test my own data?

Can this be done on Apple Silicon, I have M2 with 24G memory?

What GPUs were you running?

which machine is recommended for fine-tuning LLAMA? windows?

Hello I am getting this error can someone please help me out with it: ValueError: Batch does not contain any data (`None`). At the end of all iterable data available before expected stop iteration.

I added push_to_hub command but it didnt push.. how do I use it to test the output?

Please subscribe and like the video to help me keep motivated to make awesome videos like this one. :)

@arpitghatiya7214

10 ай бұрын

Please make a video on Llama2 + RAG (instead of finetuning)

I did the same with LLama-2, but --push_to_hub doesn't push at all.

Thanks for the tutorial! How would you modify the code to train on a dataset containing a single column of text? i.e. trying to perform domain-specific additional pretraining? I would remove the peft portion to do full finetuning, anything else?

@sanjaykotabagi4407

11 ай бұрын

Hey, Can we connect. Even I need help on similar topic. We can discuss more ...

@user-bq2vt4zz2e

10 ай бұрын

Hi, I'm looking into something similar. Did you find a good way to do this?

Hey! Fantastic video, but i do not understand at all the division into smaller chunks of the model in order to work in free version of collab, can you explain it? Thank you!

@abhishekkrthakur

11 ай бұрын

chunks are loaded into ram first. since larger chunks didnt fit in ram with all the other stuff, i created a version with smaller shards :)

Thanks for the nice tutorial. Could you show how to perform inference the finetuned model?

@abhishekkrthakur

11 ай бұрын

yes. its in one of my previous videos :) thanks!

@BrusnickiRoberto

5 ай бұрын

@@abhishekkrthakur Which one?

Any chance you have the python scripts available somewhere? Thanks in advance.

Great work Abhishek Thakur, it will be great if you made a video on how to run the fine-tuned model

@abhishekkrthakur

9 ай бұрын

already done. check out other videos on my channel

@AnandMoorthyJ

8 ай бұрын

@@abhishekkrthakur can you please post the video link? there are many videos in your channel, it's hard to find which one you are talking about.

@devyanshrastogi

8 ай бұрын

​@@abhishekkrthakur I did fine tuning on the model, but I don't think I can run it on google colab with T4 since its show out of memory error!! Any suggestion?

@ozzzer

2 ай бұрын

@@AnandMoorthyJ did you find the video? im looking for the link aswell :)

@vinodb4339

17 күн бұрын

​@@AnandMoorthyJare you able to run it?? Did you find the video link?

Very good tutorial. On what machine are you running this? I am trying to run it on a Macbook pro M1 but I keep getting "ValueError: No GPU found. Please install CUDA and try again." I have tried to do this directly on Huggingface and got "error 400: bad request"...so I cloned autotrain and ran it locally...still getting error 400. Do you have any pointers?

@nirsarkar

9 ай бұрын

Same error

can i train llama3 also with these steps?

How can I turn this into a gptq version after finetuning?

How do I use a dataset in the huggingface hub?

Thank for the tutorial! How can I fine-tune the language model with a domain-specific unlabeled dataset to improve performance on that specific domain? Is it effective to leave the instruction and input empty and only use domain-specific text for the output?

@sanjaykotabagi4407

11 ай бұрын

Hey, Can we connect. Even I need help on similar topic. We can discuss more ...

Thanks! Is there a way to push the trained model to hub after its trained and not using the --push_to_hub while training? Also, when I try to use push to hub, I get a "you don't have rights to create a model under this namespace". I am using a read token to access the llama model. Do I need to change it to a write token? Is it possible to use two separate tokens? (sorry, I'm super new to Huggingface) Any help is much appreciated. Thanks!

@abhishekkrthakur

11 ай бұрын

yes. you need to use a write token. you can remove push to hub and then push the model manually using git commands if you wish

how do you get access to the llama model?

Is there a way to run it on CPU? Thanks sir, I love your work

Hi , nice tutorial, could you please help me with this error ? , when I try to train the model I got this error : raise ValueError("No GPU found. Please install CUDA and try again.") , Do you know how to solve this ?

Why do we need --mixed-precsion and --quantization both? Aren't they both doing the same thing?

hey Abhishek, I am a full stack developer and interested in AI. I love to code. I tried learning DS but found no interest in juggling with data. How should i learn?

Can i train it with json dataset?

I am trying to use the generated model using autotrain from_pretrained ,, but its giving me error does not appear to have a file named config.json. Does anyone have the code for predicting or help me with this issue

How to change this for tasks like classification?

How can we train the model with gpu?

what is your machine configuration?

Is it possible to train with a pdf? Thanks.