How to use IPAdapter models in ComfyUI

Everything you need to know about using the IPAdapter models in ComfyUI directly from the developer of the IPAdapter ComfyUI extension.
👉 You can find the extension "ComfyUI_IPAdapter_plus" on github here: github.com/cubiq/ComfyUI_IPAd...
00:00 Introduction
01:33 Basic Workflow
05:55 IPAdapter Plus Model
07:32 Prepping Images
09:16 Sending Multiple Images
14:07 Face Model
15:18 SDXL
17:42 Img2Img
19:10 Inpainting
20:00 ControlNet
21:40 Upscaling
23:17 Saving embeds
26:33 Conclusions
🎵 Background Music
-- "Part A" by Alexander Nakarada (www.serpentsoundstudios.com)
Licensed under Creative Commons BY Attribution 4.0 License
-- Last Stop Synthwave by Karl Casey @ White Bat Audio (whitebataudio.com/)
-- CyberPunk City by Peritune (peritune.com/blog/2020/05/22/...)

Пікірлер: 260

  • @AB-wf8ek
    @AB-wf8ek7 ай бұрын

    Just want to say Thank You Thank You Thank You, from the bottom of my heart. There are very few developers that take the time to actually explain their tools, let alone include additional options such as the saving embeddings, that offer huge potential for sharing, and extending the workflow in regards to resource management. You are a huge asset and very much appreciated.

  • @ronnykhalil
    @ronnykhalil6 ай бұрын

    Your comfy node (and this video) are invaluable resources! Thanks so much for helping me wrap my head around IP!

  • @carstenli
    @carstenli8 ай бұрын

    Incredibly valuable tutorial. Keep up the good work.

  • @user-kw3ch2xr6h
    @user-kw3ch2xr6h5 ай бұрын

    this is so absolutely bonkers I can't beleive how much things have changed in 3 months. I just found your channel and binged your last months content and holy whattheheck this is brilliant.

  • @SearchingForSounds
    @SearchingForSounds6 ай бұрын

    Oh man, this is changing my world. So much we can do with this. And... you explained your tools. Thank you so much!

  • @sergeysokolov6657
    @sergeysokolov665711 күн бұрын

    i been sure i'll never understand how all this things work, especially inside of comfyui, you're just the best in your explanations, they're so clear! thanks for the knowledge you're sharing to us.

  • @matthewkolb9950
    @matthewkolb99504 ай бұрын

    Excellent video! You have such a pleasant style of communicating that it really was a pleasure absorbing all of this information. Well done and thanks!

  • @kleber1983
    @kleber19835 ай бұрын

    So much content in a single video, this is amazing... thanks so much!

  • @dasomen
    @dasomen8 ай бұрын

    What a useful tutorial, absolutelly fantastic, thanks a lot!

  • @aayushporwal6281
    @aayushporwal62813 ай бұрын

    Just want to say Thank you. For 2 days, I have been searching the way to inpaint using image and in the video, you have explained it. in a very easy way to understand. Thank you very much.

  • @SsicKsSicK
    @SsicKsSicK6 ай бұрын

    Thank you so much for this AMAZING feature and also the detailed readme plus VIDEOS! We need more people like you! You are an enrichment to the AI / Open Source community

  • @VincentRie

    @VincentRie

    4 ай бұрын

    double down on that. thank u so much for this amazing piece of work.

  • @pedrogorilla483
    @pedrogorilla4837 ай бұрын

    Thank you for creating this implementation! Very clever solutions for handling workflows! I saw you added weighting options for the images as someone requested it, I was doing it through repeating the same images a few times to increase their weight, it was very messy 😅

  • @WhySoBroke
    @WhySoBroke8 ай бұрын

    Since you have so much expertise and knowledge in this topic, I really look forward to the training model tutorial 😊

  • @latentvision

    @latentvision

    7 ай бұрын

    I'll work on that but it's really for kinda edge scenarios and optimization (I guess can be useful for some art styles)

  • @dflfd
    @dflfd5 ай бұрын

    thank you, this is fantastic -- very well explained. The saving embeds is brilliant!

  • @johnmcaleer6917
    @johnmcaleer69177 ай бұрын

    Wonderful tutorial...very clear and easy to follow...👍🏻

  • @i.venture
    @i.venture3 ай бұрын

    New to ComfyUI. Thanks for this. It was very helpful for someone like me who has heard about IPAdaptor, but had no clue what it really does.

  • @jameslanman
    @jameslanman3 ай бұрын

    Thank you for your time and energy on this. This was a great introduction to comfyui.

  • @manunoyonify
    @manunoyonify7 ай бұрын

    Awesome. Adding different image ratio inputs and outputs and the ability to give custom weights to batched input images would be a blessing !!! Thx for ur work !

  • @brandonflores4

    @brandonflores4

    3 ай бұрын

    i think scott detweiler made a video on weighted inputs. one of his comfyui episodes. unsure if it had to do with ipadapter.

  • @yiluwididreaming6732
    @yiluwididreaming67328 ай бұрын

    One of the best nodes I've seen for COMFY. Using it to lead renders with my current workflows and results show increased accuracy and detail. SUBBED!!

  • @lucretius1111
    @lucretius11116 ай бұрын

    Just came across your amazing tool. Congratulations and thank you! Amazing applications for this in the future, I think.

  • @adelechelmany
    @adelechelmany8 ай бұрын

    Man, I can't thank you enough for this, Bravo. 👏👏👏

  • @stephantual
    @stephantual4 ай бұрын

    This a game changer and continue to a be game changer. Not to mention you are kind enough to provide not just a video, but A GREAT video on how to use your tool! Thank you a million times.

  • @EH21UTB
    @EH21UTB8 ай бұрын

    Wow! Fantastic video, I learned so much, thank you!

  • @fedoborod8890
    @fedoborod88906 ай бұрын

    This is AMAZING! Your explanation and trick, omg! I learned alot!

  • @Tj4ckinator
    @Tj4ckinator7 ай бұрын

    Great video! So informative and straight to the point 👍 I would love to see your video on the training

  • @simonrobson615
    @simonrobson6154 ай бұрын

    Brilliant, thanks so much. Your system and explanation is awesome, I've learnt so much!

  • @obscuremusictabs5927
    @obscuremusictabs59275 ай бұрын

    Amazing. Thank you for all your hard work.

  • @WhySoBroke
    @WhySoBroke8 ай бұрын

    Amazing tutorial!! Much respect ❤️🇲🇽❤️

  • @euggraph1048
    @euggraph10485 ай бұрын

    Great Job explaining everything , Thank You!!!

  • @drosselmir
    @drosselmir2 ай бұрын

    I'd classify this as the top 5 informational comfy/SD video to watch. Thank you Mato sir! Also looking forward to the training tutorial.

  • @piratediffusion
    @piratediffusion6 ай бұрын

    Amazing work! Very inspirational

  • @creativojavier
    @creativojavier5 ай бұрын

    Grazie mille for sharing this tool and explaining it so clearly!

  • @arshartcloud3374
    @arshartcloud33743 ай бұрын

    Amazing! so many useful tools in one video

  • @musty5551
    @musty55518 ай бұрын

    Amazing work bud!

  • @LTE18
    @LTE187 ай бұрын

    Thank you. This is huge to the community

  • @enriqueicm7341
    @enriqueicm73415 ай бұрын

    This was very useful and very well explained! thanks you a lot!

  • @aliyilmaz852
    @aliyilmaz8522 ай бұрын

    great explanation, and very good benefits you add into ipadapter. Thank you so much

  • @CPDCT
    @CPDCT7 ай бұрын

    thanks for creating this! Game changer on comfyui!

  • @resh6701
    @resh67018 ай бұрын

    It worth 30 min without hesitation

  • @parlabaneisback
    @parlabaneisback6 ай бұрын

    A fantastic presentation, thanks so much.

  • @reapicus557
    @reapicus5572 ай бұрын

    Thank you very much for this tutorial! This tool is very powerful, and it is going to make my workflows so much easier to construct.

  • @enthuesd
    @enthuesd6 ай бұрын

    spectacular presentation, thank you

  • @techeman369
    @techeman3695 ай бұрын

    Great Explaination simple crystal clear thank you so much

  • @swannschilling474
    @swannschilling4748 ай бұрын

    Thanks a lot for this one!! Great tutorial!!

  • @ryanontheinside
    @ryanontheinside4 ай бұрын

    thank you so much yo! you are, literally, incredible.

  • @psionicxxx
    @psionicxxx5 ай бұрын

    Marvelous, keep up the good work.

  • @EliSpizzichino
    @EliSpizzichino8 ай бұрын

    Bravo! Fantastic work with IPAdapterPlus, I would be very interested to see also a video on the training process you mentioned. I'm trying to train a style that is quite unique, so I can't just use one image. I'm getting poor result with Lora standard network training, and standard dreambooth training

  • @Ekopop
    @Ekopop5 ай бұрын

    thanks a lot for your very clear explanations and this awesome tool.

  • @Sebastian-cn8lh
    @Sebastian-cn8lh6 ай бұрын

    incredible detailed!... thank you !.

  • @ALatentPlace
    @ALatentPlace8 ай бұрын

    Great extension and great tutorial! Awesome! Thanks for this It was posted on my Discord yesterday after I released a video about a more basic noise injection technique just using nodes. I will defintely try this out and, if it's ok for you, introduce it to my german speaking audience.

  • @latentvision

    @latentvision

    8 ай бұрын

    hey thanks! I checked your videos, I don't speak German but they are really well done and easy to follow. Absolutely take whatever you want from my video and the content on my repository.

  • @alpaykasal2902
    @alpaykasal29024 ай бұрын

    WOWWW! This looks amazing... I'm going to try this out tonight (I may not get any sleep this weekend :) )

  • @steventapia_motiondesigner
    @steventapia_motiondesigner7 ай бұрын

    Thanks so much for IP Adapter its been working nicely in Automatic 1111. I still have to learn how to use it thoroughly. More tutorials would be appreciated!

  • @hakandurgut
    @hakandurgut7 ай бұрын

    Very good tutorial, thanks

  • @Inner-Reflections-AI
    @Inner-Reflections-AI8 ай бұрын

    Thanks for the great guide!

  • @SamBeera
    @SamBeera6 ай бұрын

    Excellent, you are awesome, and thanks very much for the explanation and video.

  • @SamhainBaucogna
    @SamhainBaucogna8 сағат бұрын

    The best! 🤝👏

  • @M0K0K0
    @M0K0K04 ай бұрын

    Thank you very much for this video and nodes !

  • @TheBestgoku
    @TheBestgoku3 ай бұрын

    thank you soo much, clear and to the point.

  • @joonienyc
    @joonienyc5 ай бұрын

    just beginning with it and already seem this great nodes

  • @Jan-jf4th
    @Jan-jf4th5 ай бұрын

    Great video, thank you!

  • @juanmariaolaizola1858
    @juanmariaolaizola18582 ай бұрын

    really really nice, many thanks!

  • @aleksanderk5695
    @aleksanderk56957 ай бұрын

    Many thanks for Your work!

  • @rainerzufall1868
    @rainerzufall18687 ай бұрын

    So useful! Thanks!!

  • @hamedsadeghizadeh6660
    @hamedsadeghizadeh66604 ай бұрын

    Nice work, thanks

  • @svenhinrichs4072
    @svenhinrichs40726 ай бұрын

    This is magic! Thank you very much...

  • @pk.9436
    @pk.94367 ай бұрын

    Great Work please keep going!

  • @godpunisher
    @godpunisher8 ай бұрын

    nice one, i'm going to use it. thank you.

  • @igorkotov8937
    @igorkotov89372 ай бұрын

    Thank you!

  • @hilbrandbos
    @hilbrandbos7 ай бұрын

    where do you get the clipvision models that you use?

  • @angelotsk3173
    @angelotsk31738 ай бұрын

    woow very nice, i wish you make more videos and tutorials .. Thanks thanks

  • @xorq
    @xorq7 ай бұрын

    More videos like this please !!!

  • @Pat-1000
    @Pat-10005 ай бұрын

    Thank You

  • @ac3d657
    @ac3d6575 ай бұрын

    Thankyou, You do a much better job at explaining compared to a person paywalling content behind patreon. Thanks for your work on ipadapter its another indespensible tool and hopefully others will help and work toward further amazing improvements to the whole Stable Diffusion scene, and less paywalling content on youtube and such and more open source and less stagnation.

  • @latentvision

    @latentvision

    5 ай бұрын

    thanks for you message, I feel the same way. Mind you I don't think there's anything wrong in asking compensation for quality content, but since I developed an Open Source tool I find it's only fair that I also share the know-how on how to use it. I guess it's the only way we can actually evolve.

  • @ac3d657

    @ac3d657

    5 ай бұрын

    @@latentvision thankyou for your response. Correct, people wouldn't be able to learn and only watching a couple of your videos and you have made understanding comfy in general a lot easier - support chains and donations are fine but when they hide what is essentially open and free infomation found in the same videos like yours is very concerning in this community and only invites stagnatioin in the ai/stablediff space.

  • @cocopops733
    @cocopops7333 ай бұрын

    great stuff

  • @falconbmstutorials6496
    @falconbmstutorials64966 ай бұрын

    Your voice (I don't know if it is your own) sounds a lot like "my name is Giovanni Giorgio" 😊 Thanks so much for your very calm way of explaining and naturally also for your time & energy invested into the development of IPAdapter!!!

  • @yashpatidar.8506
    @yashpatidar.85068 ай бұрын

    Great Job

  • @haishengchi3333
    @haishengchi33336 ай бұрын

    this tutorial is best

  • @rohitanshu
    @rohitanshu7 ай бұрын

    @latentvision Is there a way to know how the image is being described internally? I mean, can that text be extracted somehow?

  • @imtaha964
    @imtaha9645 ай бұрын

    thank u man

  • @alamondefield2823
    @alamondefield28237 ай бұрын

    TY me and my ComfyUI loves you

  • @user-hg4hg5ix7f
    @user-hg4hg5ix7f6 ай бұрын

    bro sei un grande

  • @MyWhyAI
    @MyWhyAI5 ай бұрын

    Great tutorial! How were you able to get such high denoise on the upscale? Anything over .2 for me starts to change the look.

  • @bkdjart
    @bkdjart4 ай бұрын

    Thanks for the tools and tutorials! One question though is where can I find the clip encoders to download? I am looking everywhere and I can't seem to find the IPAdapter SD15 clip encoder anywhere. Also once I download it where do I put it so it shows up correctly? Additionally is there a way to find out the underlying path of where each node is looking for?(It would make pathing so much easier when manually adding files)

  • @frazuppi4897
    @frazuppi48975 ай бұрын

    Ciao matteo, grazie

  • @pschoro
    @pschoro7 ай бұрын

    COngrats !!!

  • @aerofrost1
    @aerofrost16 ай бұрын

    How much does the model affect the outcome? Also, how do you resize the latent image in the img2img section? I'm on a 4GB GPU so I can't run high resolution images, it will take me ages. I need to resize the images that are latent. Also the latent image does not maintain the lines like yours, yours is almost like using controlnet canny, it keeps everything the same and just changes the style, mine doesn't. It changes the entire look of the image, pose, etc.

  • @Sergatx
    @Sergatx7 ай бұрын

    Where do you get the ipadapter encoders? can't seem to find them

  • @2PeteShakur

    @2PeteShakur

    6 ай бұрын

    "Additionally you need the image encoders to be placed in the ComfyUI/models/clip_vision/ directory: SD 1.5 model (use this also for all models ending with _vit-h) SDXL model"

  • @alecubudulecu
    @alecubudulecu8 ай бұрын

    Very helpful. Is IPadapter similar to A4’s “reference” controlnet?

  • @elowine
    @elowine7 ай бұрын

    I'm having a hard time finding the clipvision model, did you use the 'h94/IP-Adapter' one on huggingface? Thx!

  • @latentvision

    @latentvision

    7 ай бұрын

    yes they are both in the h94 repository on huggingface. the "sdxl" one is good only for one model (the base sdxl). for all others you use the vit-h

  • @elowine

    @elowine

    7 ай бұрын

    @@latentvision Thanks for developing this tool! So much easier to work with IPAdapter when its not just a commandline and a prompt

  • @AnnisNaeemOfficial
    @AnnisNaeemOfficial2 ай бұрын

    I love this tutorial. But at 14:21, you have a load clp vision: ipAdapter_image_encoder.sd15.safetensors. I have been looking everywhere for this image encoder but cannot find it. I can only find the clipvisionG or the ViT-H clip tensors. Any tips?

  • @jerr.___.y
    @jerr.___.yАй бұрын

    Where can I get the ip-adapter_sd15.bin file for the IP adapter model loader node and in which folder should I put it?

  • @whipbeats
    @whipbeats5 ай бұрын

    I cant find the image encoder sd15 safetensors, does someone know where i can download it ? thanks

  • @bradens8584
    @bradens858419 күн бұрын

    thank tyou

  • @frazuppi4897
    @frazuppi48975 ай бұрын

    how do you create the mask from the UI?

  • @jasemali1987
    @jasemali19875 ай бұрын

    Ladies and gentlemen, we have found THE GUY ! Thank you so much for everything you are doing to the Ai community, this is great work and a great explanation. May I ask what is the limitation of the IP adaptor in terms of the fed images, like on what bases it determines the tokens? For example, if i used a line art image with a realistic checkpoint trained for producing mostly photographic images, will the IP adaptor get the tokens from the trained checkpoint or from the IP adaptor model ?

  • @latentvision

    @latentvision

    5 ай бұрын

    thanks for the kind words. IPAdapter is a very strong conditioning but the main checkpoint will always show its character. It's better to pick the right model for the image you wish to generate

  • @nodewizard
    @nodewizard5 ай бұрын

    Truly amazing. I would give you a Noble Prize if I could. Thank you Matteo. For upscaling with IPAdapter, why not send it through a ControlNet Tile along with an Ultimate Upscaler? You could create an image in an SDXL checkpoint, then IMG2IMG with ControlNet tile in a SD1.5 checkpoint. The denoise of the tile model should be 0.3 or less to give the closest results to the image.

  • @latentvision

    @latentvision

    5 ай бұрын

    hey thanks the point of that segment is to show you the strength of the IPAdapter model. Of course you'll have to mix it with other nodes in a real life scenario. If you use tile controlnet already though you probably don't need to add IPAdapter

  • @latent-broadcasting
    @latent-broadcasting5 ай бұрын

    Thanks so much! I applied this combined with Kohya DeepShrink and the results are amazing. Just a question, If I want to take the face using the face adaptor and I want it to combine it with an adapter that takes the style of the photo, what should I do? I can't find a workflow for that

  • @latentvision

    @latentvision

    5 ай бұрын

    you can daisy chain IPAdapters, check my attention masking video, you can see how it works there

  • @latent-broadcasting

    @latent-broadcasting

    5 ай бұрын

    @@latentvision Thanks for your answer! I'll check that video

  • @TheClemt33
    @TheClemt337 ай бұрын

    Hi thank you so much for your wonderful work ! Question : How can we have clear face replacement ? Now it's merging faces together. Not the most accurate output, the style is preserved, but face details are merged.

  • @latentvision

    @latentvision

    7 ай бұрын

    I'm not sure that is feasible with the current implementation, unless you do some extensive training I guess. This is not a "face swap" but a "face describer" Be sure to use a closeup portrait of the person and use the prepimage node adding a little sharpening if needed.

  • @xq_le1t0r97

    @xq_le1t0r97

    7 ай бұрын

    Just use re-actor face component... do the image you want.... and add the end just replace the face. Don´t use a photo of the person you want as base for the IPA.

  • @yosribengaidhassine9299
    @yosribengaidhassine929915 күн бұрын

    Hello , I have installed Ip adapter in custom nodes when I have tried to add node I don't found apply Ip adapter

  • @SparkFlowAAA
    @SparkFlowAAA5 ай бұрын

    Have a question. In SDXL section you've put SDXL encoder, but in documentations it says to put SD1.5 encoder? Which one is correct?

  • @latentvision

    @latentvision

    5 ай бұрын

    check the documentation, one SDXL model requires the SDXL encoder, all others work on the SD1.5 encoder