How To Do Stable Diffusion Lora Training In ComfyUI (Tutorial Guide)

Ғылым және технология

How to use the new Comfy UI custom nodes for effortless Lora training. In this video, we walk you through the installation process and demonstrate how to train Lora easily using the provided custom nodes. With Comfy UI, you can simplify the process and save valuable time.
Resources For ComfyUI Lora Training : / 100723165
Lora Training Using Kohya-ss and Colab By Dave : • Stable Diffusion Train...
The tutorial covers the essential steps, including installing the Lora training Comfy UI custom nodes from their GitHub page. We explore both the simple and advanced modes, showcasing the range of options available for Lora training. Additionally, we highlight the image captioning custom nodes and guide you through the installation process for generating image-to-text prompts.
Throughout the tutorial, we provide a step-by-step demonstration of the workflow, including loading captions, showing text, and saving text strings in the desired destination folder. We also discuss the WD 1.4 custom nodes, which enable image processing as text prompts, and explain how to connect them in the workflow.
By the end of this tutorial, you'll have a clear understanding of how to use Comfy UI custom nodes for Lora training and image captioning. This time-saving method will revolutionize your Lora training process, making it more efficient and user-friendly.
Join us as we explore the power of Comfy UI and unlock the potential of Lora training. Don't miss out on this easy-to-follow tutorial that will enhance your workflow and boost your productivity.
If You Like tutorial like this, You Can Support Our Work In Patreon:
/ aifuturetech
Discord : / discord

Пікірлер: 58

  • @amorgan5844
    @amorgan58442 ай бұрын

    Good stuff! I watched at least 20 vids yesterday, and none were clear and thorough. You, sir, deserve more subscribers

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    Appreciate for your support , hope this video help you guys. :)

  • @amorgan5844

    @amorgan5844

    2 ай бұрын

    ​@@TheFutureThinkercrap, never mind. The new images were jpg and not png.

  • @xcom9648
    @xcom96482 ай бұрын

    Thanks for always giving us the latest methods, i'm gonna try and see if i can improve my LoRa's with this.

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    Good luck! And have fun with Lora :)

  • @kalakala4803
    @kalakala48032 ай бұрын

    Thank you for tutorial, this is more convenience to train lora.

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    Yes it is, just simplify the process of training lora model. In theory, it can also do it for CP model, i believe.

  • @Halfgawd_Halfdevil
    @Halfgawd_Halfdevil2 ай бұрын

    This node does not work. I can not get it past the first step then it stops and will not work again. Any reason for this? is there a specific model you have to train in? doesent seem very useful at all if it only works with one model or not at all which seems to be the case for me. I have watched many tutorials on this and none address this issue

  • @NgocNguyen_NDN
    @NgocNguyen_NDN2 ай бұрын

    Introducing an Advanced node and Access Tensorboard node! Access Tensorboard is a very simple node that launches a URL to see data about the logs created during training. Click the node once (after, during, or even before training!), then copy-paste the URL that it wrote in the command prompt. The logs folder related to Tensorboard is in the same folder as your launcher file. (I don't understand this part and where to copy the URL, can you help me with this part?)

  • @iresolvers
    @iresolversАй бұрын

    I have it working at 5 epochs and 10 batch but I keep geting out of memory errors when I go to a batch count of 100 or more not sure why I have an RTX 3090 24GB ov vram

  • @guillermoauditoree815
    @guillermoauditoree8152 ай бұрын

    Hi bro, i love you work,have a great day, i have one question ,This method work on amd gpu?

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    It depends you Comfyui setup. I haven't play with AMD GPU yet. So I can't answer you accurately

  • @alexandretamisier8150
    @alexandretamisier81502 ай бұрын

    Hey, thanks for your explanation. But I struggle on something when I try on my end. I've created a clean Comfy UI install, just installed "Lora-Training-in-Comfy" but at the restart moment, I get the error "Torch not compiled with CUDA enabled". I don't figure how to get rid of it with the info I found on the internet. Do you have an idea of what I should do ? Have a good day

  • @vibeactvst8686

    @vibeactvst8686

    2 ай бұрын

    I had the same issue, saw a lot about ensuring you have pytorch and cuda 12.1 which is probably correct however I couldn't get it to work until I did a fresh comfyui install and moved the comfyui portable folder. It has to be the first folder in the path like this E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy I would try this just to make sure, the issue is with a folder in the huggingface folder with a very long name during install

  • @alexandretamisier8150

    @alexandretamisier8150

    2 ай бұрын

    @@vibeactvst8686 Hey man, i've tried some random things based on your comment and thanksfully it worked after some attempt. Thank you !

  • @vibeactvst8686

    @vibeactvst8686

    2 ай бұрын

    @@alexandretamisier8150 Happy it helped somehow. Now I am having issues with rust compiler/cliptextmodel/transformers One of those isn't letting me training the lora 🤦🏿‍♂️

  • @alexandretamisier8150

    @alexandretamisier8150

    2 ай бұрын

    @@vibeactvst8686 Same thing, i don't know the reason but the lora training stop instantly when i want to run it :( i've tried to install Kohya too and get errors, i think i should learn python before trying this haha

  • @kleber1983
    @kleber19832 ай бұрын

    what about the files preparations? file sizes, nomenclature, proportion etc anything goes? I miss some of that infortmation. thx.

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    it's called Dataset. SD 1.5, and SDXL image requirement are difference, it's 512X512 and 1024X1024 px. I don't explain everything about Lora, this video are only focus on this custom node. If I need to explain everything about Lora, it will need a hour long video, and I don't think you have the patient to watch those. ;) cheers.

  • @kleber1983

    @kleber1983

    2 ай бұрын

    @@TheFutureThinker Well then this video is not about "How To Do Stable Diffusion Lora Training In ComfyUI" but instead it's how to understand these custom nodes for people who already have some baggage in lora training.

  • @unknownuser3000
    @unknownuser30002 ай бұрын

    OK, you sold me on comfy. I've been training with kohya dreambooth and onetrainer and love trying new training tools. I can't get sdxl to train in onetrainer at all but I'm only on a 3080 so maybe the vram is lessened here.

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    I used Kohya dreambooth to lora lora and checkpoint before. And when i saw this custom node, i feel it is very convenience

  • @galdrencyphr

    @galdrencyphr

    17 күн бұрын

    @@TheFutureThinker you said after training in the end how do we train? I do not know what to click to train 😅

  • @TheFutureThinker

    @TheFutureThinker

    16 күн бұрын

    @@galdrencyphr click Queue Prompt in Comfy as usual to execute workflow

  • @HiProfileAI
    @HiProfileAI2 ай бұрын

    Did anyone get it to eork with SDXL models? Gonna give it a try this week.

  • @RuinDweller
    @RuinDweller2 ай бұрын

    Thank you very much for the tutorial, but this does not work for me. :( I can make the text files for the images, but "training" takes 17 seconds, and no file is produced in my loras folder. I get no error, the terminal just says it was completed, but there's no file there. Can you tell me what I'm doing wrong? This should be simple, but it's NOT. :(

  • @Halfgawd_Halfdevil

    @Halfgawd_Halfdevil

    2 ай бұрын

    This node has never worked for me. every tutorial seems to make it look easy but there is not really detail ever shown of the actual process. I see no usable models ever demonstrated after creation by any of the posters of these videos. maybe someone will actually explain why it fails for so many people.

  • @RuinDweller

    @RuinDweller

    2 ай бұрын

    @@Halfgawd_Halfdevil I finally got it to work by making sure that the folder that held my database folder was at the root of its directory ...IE, "D:folder/database folder/". If you try to train the files to anywhere else, it won't dork ...or I guess I should say it didn't work for me. Also make sure that you name the main folder "database", and name the folder with your actual database in it "5_s" and then the name of your LoRA, with no space in between. So, my folder was called "5_sNPC", because the name of my LoRA was "NPC". By the end it was set up like this: "D:database/5_sNPC/". Hope this helps!

  • @Halfgawd_Halfdevil

    @Halfgawd_Halfdevil

    2 ай бұрын

    @@RuinDwellerGlad you got it to work! I have all the database folders etc. but I was using "5_" without a lowercase s. I will give that a try. I have no real issues with the tagger saving to the database folder with only the 5_ before the model's name so not sure why one would work and not the other, but I will give it another try. Thanks for the info. I appreciate it. BTW how are the Lora's working? Do they take long to train...so I have a reference for time I should be waiting? Is It still sd1.5 only? just curious as I use almost exclusively SDXL so If it isn't going to train in SDXL it's really pointless for me anyway.

  • @RuinDweller

    @RuinDweller

    2 ай бұрын

    @@Halfgawd_Halfdevil I didn't try with XL simply because it has a higher VRAM requirement, it's even slower than 1.5, it already took me 20+ hours to train 75 epochs, and it scraped the ceiling of my VRAM the whole time. ...But it's possible to do! You just have to have a much better system than I currently have. Did you get this to work? I did get mine working, but even after all that training, they still aren't really "crisp" or photographic, but I probably need to turn the training up. That's another thing that's missing from all of these tutorial videos. The settings they show before training SIMPLY CANNOT be the ones they actually used for it, there's no way.

  • @leonardkim690
    @leonardkim6902 ай бұрын

    Thanks for your tutorial. but I've got an error. 'C:\StabilityMatrix-win-x64\Data\Assets\Python310\python.exe: Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')' , could you let me know how to fix it?

  • @NgocNguyen_NDN

    @NgocNguyen_NDN

    2 ай бұрын

    Have you recovered yet?

  • @ki6o

    @ki6o

    2 ай бұрын

    Hello, same for me! Not sure what my options are?

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    Why is your Python running in Stability Martix? Seems like it doesn't work with Martix installation

  • @user-dp8bj4ol8i

    @user-dp8bj4ol8i

    2 ай бұрын

    Is there any solution?

  • @theanhtran6848

    @theanhtran6848

    Ай бұрын

    hi, same for me, and i watch many time from video of owner (Larry Jane). but It still not work, if anyone have solution, i will I would be very grateful.

  • @indexlibrotorium1217
    @indexlibrotorium1217Ай бұрын

    I get this when i try i use the trainer Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')

  • @testingbeta7169
    @testingbeta71692 ай бұрын

    also need result demo

  • @tonon_AI
    @tonon_AI2 ай бұрын

    tried installing your nodes and my comfyUI crashed and don't open anymore, will have to reainstall everything...

  • @TheFutureThinker

    @TheFutureThinker

    2 ай бұрын

    1- That is not my node, please clarify. 2- if you follow the installation, it wouldn't be a problem. 3- Crash Custom Node can be deleted in folder and ComfyUi reopen again. It can run normal. Shame so called AI on your name, and don't know how to do troubleshooting.

  • @tonon_AI

    @tonon_AI

    2 ай бұрын

    @@TheFutureThinker I downloaded the workflow, installed the missing nodes in the manager, clicked om restart and it just got stuck, I closed and open again, and it stops working, comfy dont start anymore, I updated everything but nothjng. Ill just reinstall again from zero.

  • @LuckRenewal

    @LuckRenewal

    2 ай бұрын

    ​@@tonon_AIit happens sometimes. It's weird like that. So best look at issues page before installing new nodes to see what problems others have

  • @Marcus-si7su

    @Marcus-si7su

    2 ай бұрын

    "Found a solution that worked for me and other people as well." you might try. Go in to your ComfyUI_windows_portable then Update and use the Update comfy and python. Also took away this Node for the Training of Lora.

  • @Royvcent

    @Royvcent

    2 ай бұрын

    Yup the installation of the first 'Lora-Training-in-Comfy' and restart after also completely wrecked my Comfyui. I was unable to start it. Gives the following message in the terminal: "AssertionError: Torch not compiled with CUDA enabled." Deleting the crash custom node folder doesn't fix the problem. EDIT: Updating with 'update_comfyui_and_python_dependencies' in the Comfyui>update folder fixed it again.

  • @collectiveunconscious3d
    @collectiveunconscious3d21 күн бұрын

    I keep getting an error that the node in costom-nodes/huggingface/__init__.py doesn't exist, I searched everywhere but I cant find if there's a thing as this custom node. I've also installed the requirements but it doesn't install this things. Heeeeelppppp

  • @collectiveunconscious3d

    @collectiveunconscious3d

    21 күн бұрын

    Okee, I fixed that one by putting an empty __init__.py in there - Up to the next error lol

  • @TheFutureThinker

    @TheFutureThinker

    21 күн бұрын

    Um.. in my setup, I haven't use Huggingface node.

  • @collectiveunconscious3d

    @collectiveunconscious3d

    21 күн бұрын

    @@TheFutureThinker I've tried about everything cant get it to work. Installed torch more times then I can count, tried in vr anaconda. Checked all the cuda stuff with different version... Alwayst get error related to some dll or some ini file. I give up lol, wasted way too much time

  • @TheFutureThinker

    @TheFutureThinker

    21 күн бұрын

    Yes, if this don't work , just use Colab Lora training or Kohyass. Some open source project is like that, they created it but not compatible for every system.

  • @collectiveunconscious3d

    @collectiveunconscious3d

    20 күн бұрын

    @@TheFutureThinker Yeah thanks for the video though, Im using Kohya now, seems to work on my system

  • @Inkelish
    @InkelishАй бұрын

    Hi, Ive been having a lot of issues with this and I've gotten so close to fixing it all. When I start the training everything works well until I am met with this eror: raise RuntimeError('Error(s) in loading state_dict for {}: \t{}'.format( RuntimeError: Error(s) in loading state_dict for CLIPTextModel: Unexpected key(s) in state_dict: "text_model.embeddings.position_ids". I dont know how to fix this and I cant find anything online that helps.. Hope you can help me out with this! thanks! (:

  • @0AD8
    @0AD8Ай бұрын

    anybody els get stuck with the error: 'pip' is not recognized as an internal or external command, operable program or batch file. when doing the cmd: ComfyUI\custom_nodes\ComfyUI-WD14-Tagger> pip install -r requirements.txt ?

  • @asyrofmuzakki

    @asyrofmuzakki

    Ай бұрын

    try to type all: ../../../python_embeded/python.exe -s -m pip install -r requirements.txt it works for me

Келесі