Python Selenium Tutorial #6 - Bypass Detection using plugins, settings & proxies

🌐 NodeMaven Proxy Provider: go.nodemaven.com/scrape
💥 Special Bonus: Use "Michael" at checkout for an extra +2GB of bandwidth.
🤖 2captcha Captcha Solving Service: bit.ly/2captchapromo
This selenium tutorial is designed for beginners to learn how to use the Python selenium library to perform web scraping, testing, and creating website bots. Selenium is a Python library that provides a high-level API to control Chrome or Chromium and Firefox or Geckodriver over the DevTools Protocol. Selenium runs non-headless by default but can be configured to run headless.
Playlist: • Python Selenium Tutorial
Code: github.com/michaelkitas/Pytho...
Join our Discord: / discord
Undetectable ChromeDriver: pypi.org/project/undetected-c...
Gecko Driver: github.com/mozilla/geckodrive...
Chrome Driver: chromedriver.chromium.org/dow...
Download Visual Studio Code: code.visualstudio.com/download
Download Python: www.python.org/downloads/
Selenium Library: pypi.org/project/selenium/
Donate
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
PayPal: support@websidev.com
Bitcoin Wallet: bc1q05j8gcnq4mzvgj603cxdc8xxck4jgnu2ljsrt4
Ethereum Wallet: 0x5e7BD4f473f153d400b39D593A55D68Ce80F8a2e
Social
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Website: websidev.com
Linkedin: / michael-kitas-638aa4209
Instagram: / michael_kitas
Github: github.com/michaelkitas
Business Email: support@websidev.com
Tags:
- Python Selenium Tutorial
- Full Course Selenium
- Python Selenium
- Web Scraping Full Course
- Python Selenium Web Scraping Full Course
#selenium #python #webscraping

Пікірлер: 150

  • @MichaelKitas
    @MichaelKitas2 жыл бұрын

    You can also check how "undetectable" your selenium script is by visiting this website, from your script: bot.sannysoft.com/

  • @blitzed621

    @blitzed621

    2 жыл бұрын

    it doesn't work :/

  • @user-ny4ui7ow5n
    @user-ny4ui7ow5n Жыл бұрын

    man, you are creating content which doesnt exist on my country thank you.

  • @seomaster7991
    @seomaster79912 жыл бұрын

    Thank you for your tutorial Michael 👍

  • @hiimrain600
    @hiimrain600 Жыл бұрын

    Nice tutorial, thank you for this video. It really helps me in scaping web

  • @softwareengineer8923
    @softwareengineer8923 Жыл бұрын

    Thanks for a great video,it helped a lot.

  • @LifeOf97
    @LifeOf972 жыл бұрын

    This is great, thanks man

  • @rcgnetworks
    @rcgnetworksАй бұрын

    Thanks bro 😍 It worked like a charm

  • @larsabi_
    @larsabi_8 ай бұрын

    YOURE THE GOAT MY FRIEND :)

  • @epv888
    @epv88810 ай бұрын

    Thanks bro, thats what I needed

  • @keremsozen380
    @keremsozen380 Жыл бұрын

    the selenium goat

  • @kyei22ify
    @kyei22ify11 ай бұрын

    very helpful, thanks

  • @cejayrich4035
    @cejayrich4035 Жыл бұрын

    Thanks alot, great work

  • @thehuman5607
    @thehuman560710 ай бұрын

    Selenium expert🎉

  • @krisnadwisetyadi2964
    @krisnadwisetyadi29646 ай бұрын

    thank you for this video. it really help me. thanks man

  • @MichaelKitas

    @MichaelKitas

    6 ай бұрын

    Glad to hear that!

  • @user-el3mj4om2m
    @user-el3mj4om2m6 ай бұрын

    thanks dude it works

  • @netting.
    @netting.2 ай бұрын

    You are a LEGEND!

  • @blackcatdevel0per
    @blackcatdevel0per Жыл бұрын

    Big thanks!

  • @uplifthabesha754
    @uplifthabesha75411 ай бұрын

    It works wow, Thank you

  • @ahmedellban5748
    @ahmedellban5748 Жыл бұрын

    Thank you, you saved me

  • @user-mm4du2cd5k
    @user-mm4du2cd5k Жыл бұрын

    Thank u so much!!!

  • @simntimtwins
    @simntimtwins3 ай бұрын

    Worked for me!!! 😊

  • @parpermeter
    @parpermeter Жыл бұрын

    like bro, ty!

  • @anurajms
    @anurajms2 жыл бұрын

    thank you

  • @tuyenlamchan5927
    @tuyenlamchan59272 жыл бұрын

    nice video tks bro

  • @gayashan99
    @gayashan99 Жыл бұрын

    So useful 🙃

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Glad it was helpful!

  • @adityasprojects7126
    @adityasprojects71262 ай бұрын

    You are a god damm hero

  • @KrAsHeDD
    @KrAsHeDD2 жыл бұрын

    Thank you for this video, really helped a lot. Could you please make a video on how to use socks5 directly on selenium python. I make use of an extension to tunnel my proxy(SOCKS5) which is stressful.

  • @jw200
    @jw20011 ай бұрын

    Very good tuts, thank you! Howi can use multiple profiles? Or its better just reuse cookies? Do my stuff, clear cookies, load next account cookies and so on? Im working with 150-200 accounts

  • @MichaelKitas

    @MichaelKitas

    11 ай бұрын

    Cookies for sure then

  • @Devillunar
    @Devillunar2 жыл бұрын

    Sadly it does not pass invisible recaptcha. However, thank you for the demonstration, couldn't understand before what to change exactly.

  • @nikospomas7621
    @nikospomas76212 жыл бұрын

    Αφεντικο δεν κτλβνω χριστο απο αυτα αλλα εισαι πολυ μεγαλος μαγκας για την δουλεια αυτη

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    😂😂😂

  • @ibrahimaouladabderahman9457
    @ibrahimaouladabderahman9457 Жыл бұрын

    Please how i can run multi instances at the same time with undetected chromedriver, I tried with threads but it doesn't work

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Will make a video on that soon

  • @faaizaziz6430
    @faaizaziz6430 Жыл бұрын

    Thanks. I am using the same technique but need to run it on the server. How can I use this in headless mode?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    To run chrome-headless just add --headless via chrome_options.add_argument i.e.: from selenium import webdriver from selenium.webdriver.chrome.options import Options chrome_options = Options() #chrome_options.add_argument("--disable-extensions") #chrome_options.add_argument("--disable-gpu") #chrome_options.add_argument("--no-sandbox") # linux only chrome_options.add_argument("--headless") # chrome_options.headless = True # also works driver = webdriver.Chrome(options=chrome_options) start_url = "duckgo.com" driver.get(start_url) print(driver.page_source.encode("utf-8")) # b'

  • @Hacking-NASSA-with-HTML
    @Hacking-NASSA-with-HTML Жыл бұрын

    Hi! What proxies ipv6 or ipv4 would you recommend, sir 🤔? Which ones are more suitable?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    I think ipv6 is faster, since it’s newer but not sure

  • @DiegoAlbertoLeonLopez
    @DiegoAlbertoLeonLopezАй бұрын

    I have a doubt, could i rotate the user to have a better No-detection or it would be the same?

  • @MichaelKitas

    @MichaelKitas

    Ай бұрын

    Good question! Really depends on the particular website you trying to bypass. If you are using the same ip address for example while rotating user, usually you will get banned or at least detected. So you need a custom solution for each case.

  • @user-uw3ki8oo4e
    @user-uw3ki8oo4e4 ай бұрын

    Can you show how to bypass the linkedinnnn verfication using Selenium , please?

  • @user-ns5yp5dx5v
    @user-ns5yp5dx5v5 ай бұрын

    @MichaelKitas, my friend, tell me please. Is it updated google's code or it is problem from my side. After I put my email and click next I have a message : "This browser or app may not be secure. Learn more Try using a different browser. If you’re already using a supported browser, you can try again to sign in."

  • @MichaelKitas

    @MichaelKitas

    Ай бұрын

    It detects you are a bot, going to make an updated video on that soon. Hint: github.com/zfcsoftware/puppeteer-real-browser

  • @sniper6233
    @sniper62338 ай бұрын

    Cannot use the undetected_browser with the current selenium version. It is always giving an error that chromoptions has no headless attributes

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    Try this fix: use options = ChromeOptions() from undetected_chromedriver.v2, not selenium. Then, options.headless = False. This sets up non-headless mode, ensuring the browser window is visible. Compatibility between Selenium and undetected_chromedriver versions also matters, so double-check those if you're still facing issues!

  • @sniper6233

    @sniper6233

    8 ай бұрын

    @@MichaelKitas it worked. Thanks.

  • @ITEngines
    @ITEngines Жыл бұрын

    would you recommend selenium or js for creating checkout bots?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Puppeteer since it has some useful plugins to bypass detection and also captcha etc and it’s a lot easier to configure them in js

  • @zlukian
    @zlukian Жыл бұрын

    Lof U hahaha Greetings!!!!

  • @03nomere28
    @03nomere282 жыл бұрын

    undetected chromdriver n remote--debugging. which one is better?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Depends on the use case

  • @kelechiihunda9042
    @kelechiihunda9042 Жыл бұрын

    Nice video, please how can run Mobile emulation with undetected chromedriver?. It's doesn't seem to work with UC but works with normal chrome driver. And there's no useful information online on how to do it.

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Not sure either, but when I have time will do some research and make a video on it 👍

  • @apisn21
    @apisn21 Жыл бұрын

    can you make a playwright tutorial ? or compare selenium with playwright? Compare between headless and visible browser ? Thanks a lot, great video

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Sure, will do soon 👍

  • @mohamedbhasith90

    @mohamedbhasith90

    Жыл бұрын

    ​@@MichaelKitashi bro, how do i open specific profile in edge browser using selenium?

  • @zlwFan
    @zlwFan10 ай бұрын

    Can you show how to avoid detection using Selenium with the Firefox browse, please?

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    Good idea, yes will do a video in the near future

  • @sagarsarkar5245
    @sagarsarkar5245 Жыл бұрын

    Can you suggest how can we handle in Java Selenium ?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Not familiar with Java, but maybe soon I am going to make a tutorial on that 👍

  • @hasting8422
    @hasting8422 Жыл бұрын

    Hi! i am trying to log in to fiverr in the same way that you logged in to gmail but somehow fiverrs detection tool forces me to pass a captcha. Fiverr detects that i use selenium but i don't know how. Can you please help with how to get passed fiverrs bot detection tool

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Have you tried all methods I showed?

  • @hasting8422

    @hasting8422

    Жыл бұрын

    @@MichaelKitas Yes i have but they still block me or give a lot of captchas. I think they identify that i visit their page with selenium or maybe there is something suspicious with my user agents. I can not scrape their site and it is really annoying. Have any ideas that can help?

  • @LeandroCursinofoto

    @LeandroCursinofoto

    Жыл бұрын

    @@hasting8422 I have the same problem with another website.

  • @joel_mathew
    @joel_mathew7 ай бұрын

    can we use ur proxy server or where can we get one?

  • @MichaelKitas

    @MichaelKitas

    7 ай бұрын

    I suggest to get your proxies from this provider: nodemaven.com/?a_aid=michaelkitas It's the one I use as well👍

  • @ReverbedRetroMusic
    @ReverbedRetroMusic Жыл бұрын

    it just open and close instantly. it gives me an error the handle is invalid

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Can you copy/paste the full error, as I am not sure what "handle" it's referring to

  • @mejnuncnar9314
    @mejnuncnar9314 Жыл бұрын

    how to save result in txt file?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Check this out: stackoverflow.com/questions/33418777/write-a-line-into-a-txt-file-with-node-js

  • @saymindelilah8452
    @saymindelilah8452 Жыл бұрын

    hello how to deploy selenium on replit

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Not familiar with replit, but will check it out and make a video on that

  • Жыл бұрын

    How can I use it with robotframework?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Not familiar with it, but I am going to check it out and maybe make a video in the future

  • @mohammadaqdas7998
    @mohammadaqdas7998 Жыл бұрын

    I got an error of "Entry point Not found" Someone help please..

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Can you share code?

  • @haroldabella2041
    @haroldabella20412 жыл бұрын

    Do you know a way on how to add authenticated proxies on undetectable chrome driver?

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    It should be like you normally do: chrome_options.add_argument('--proxy-server=ip:port') driver = uc.Chrome(options=chrome_options)

  • @CodeRed-xj3ut

    @CodeRed-xj3ut

    2 жыл бұрын

    @@MichaelKitas not working with user pass proxy. By the way do you know how to save proxy with google login session?

  • @haroldabella2041

    @haroldabella2041

    2 жыл бұрын

    @@MichaelKitas how about saving proxy with google login cookies using pickle?

  • @arthurnicholas9003
    @arthurnicholas9003 Жыл бұрын

    how can i run selenium without being headless on aws ?

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    Running Selenium without headless mode on AWS (or any cloud provider) requires a virtual display, as these servers don't have a physical screen. You can use tools like 'Xvfb' to simulate a display. However, this setup can be complex and resource-intensive. It's generally more efficient and cost-effective to run Selenium in headless mode, especially in cloud environments. If you absolutely need a UI, consider using a desktop-as-a-service solution or a dedicated server with a GUI Here is a relevant tutorial I made for that: kzread.info/dash/bejne/q3in0tl6nZzFo7w.html

  • @williams9708
    @williams97082 жыл бұрын

    Can it be used in headless mode? Undetectable-chromedriver?

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    Yes, it should

  • @williamjackson1142

    @williamjackson1142

    Жыл бұрын

    @@MichaelKitas when i try in headless it gets detected, do you know why?

  • @alexdolas9911
    @alexdolas9911 Жыл бұрын

    Thanks, but this method doesnt work with chatGPT.

  • @AlexeyZubkov
    @AlexeyZubkov Жыл бұрын

    Any idea how to bypass CloudFlare with Zyte SmartProxyManager?

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    I have found how to bypass Cloudflare captcha, not sure about the one you mentioned, can you send me a url/website that uses that?

  • @AlexeyZubkov

    @AlexeyZubkov

    8 ай бұрын

    @@MichaelKitas docs.zyte.com/smart-proxy-manager.html

  • @chupapimunanyo2596
    @chupapimunanyo25962 жыл бұрын

    What is your colorscheme there? I like it

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    Bearded Theme in VS code

  • @chupapimunanyo2596

    @chupapimunanyo2596

    2 жыл бұрын

    @@MichaelKitas thanks very much :-)

  • @Felixprz17
    @Felixprz172 жыл бұрын

    is there anyway to stop it from closing after the task is done?

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    Any questions that you have the fastest way to find an answer is by “googling” so for example search: python selenium don’t close browser Here is what I found useful: stackoverflow.com/questions/51865300/python-selenium-keep-browser-open Also you can set a timeout, with time.sleep at the end of the code and make it sleep for as long as you want

  • @Felixprz17

    @Felixprz17

    2 жыл бұрын

    @@MichaelKitas thank you, I tried it but it only works sometimes on Zalando and not anytime sadly

  • @KrAsHeDD

    @KrAsHeDD

    2 жыл бұрын

    @@Felixprz17 I had this issue which made me mad because I didn't write for the program to close. If you imported unnecessary module and they are not used, it sometimes makes it close even after removing "driver.close" or "driver.quit". Try it if it works for you.

  • @yassinbousaadi1563

    @yassinbousaadi1563

    2 жыл бұрын

    I actually didn't have the problem before using the undetected chromedriver module, but it changed after that. So maybe try with webdriver.

  • @Felixprz17

    @Felixprz17

    2 жыл бұрын

    @@yassinbousaadi1563 yeah same for me. I found out that it closes as soon as it gets a error. I just set time.sleep(10) after the program is done. This way it doesn’t close instantly and I can see if it works

  • @Alex111motovlog
    @Alex111motovlog Жыл бұрын

    Can you make a video combinating this method with gspread? It would be very helpful.

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    I haven’t used gspread but I will try to make

  • @Alex111motovlog

    @Alex111motovlog

    Жыл бұрын

    @@MichaelKitas I'm excited to see a video from you about this. Especially if it is with an example extracting titles of several products from different websites and adding them in the form of a list in Google Sheets. 🙌🏻👏🏻 Success for you! I look forward to learn it from you.

  • @redouanechkerint3312
    @redouanechkerint33122 жыл бұрын

    I don't know why your Code doesn't work with me "gmail detected that selenium"

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    Make sure you follow all the steps on this video as I showed multiple versions of the code which not all are undetected

  • @redouanechkerint3312

    @redouanechkerint3312

    2 жыл бұрын

    @@MichaelKitas OK thank you so much

  • @GenaTroy
    @GenaTroy Жыл бұрын

    Please make a video on how to fake browser fingerprints, especially canvas fingerprints.

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Nice idea

  • @GenaTroy

    @GenaTroy

    Жыл бұрын

    @@MichaelKitas Yeah! That would be really useful!

  • @terryterry1655
    @terryterry1655 Жыл бұрын

    Does it work on KZread?

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    You can try, I had done it in the past, if you use your chrome and user data directory instead of chromium it should be easy

  • @user-xu2rq8qp9e
    @user-xu2rq8qp9e11 ай бұрын

    How do you set the proxy user and the proxy password?

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    Making a video on proxies very soon

  • @svitoch_blockchain

    @svitoch_blockchain

    8 ай бұрын

    @@MichaelKitas wait for this video)

  • @washingtrend9538
    @washingtrend9538 Жыл бұрын

    How do i know ip address and port

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    IP address, are usually bought from providers, or can be found for free in some website (low quality ones)

  • @swahagaming5261
    @swahagaming52613 ай бұрын

    What about tiktok sliding captcha ?

  • @MichaelKitas

    @MichaelKitas

    Ай бұрын

    Good idea, will make a video on that

  • @uplifthabesha754
    @uplifthabesha754 Жыл бұрын

    Can you Please do a video on how to bypass DataDome Anti-bot protection. Thank you

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    Few people requested that, will check it out and probably make a video for that

  • @minhnhat5914
    @minhnhat5914 Жыл бұрын

    can u share your theme name

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Bearded Theme - Arc

  • @OnlyUseMeEquip
    @OnlyUseMeEquip4 ай бұрын

    are you buying comments on your videos and then replying to them?

  • @MichaelKitas

    @MichaelKitas

    4 ай бұрын

    No, they all real 😁

  • @matrixwaves
    @matrixwaves Жыл бұрын

    bro how is this work with adsense be honest ?

  • @MichaelKitas

    @MichaelKitas

    8 ай бұрын

    Not sure what you are referring to.

  • @ataimebenson
    @ataimebenson Жыл бұрын

    I used your exact code and still got detected, maybe Google has found a way to block it

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    You can get detected by your IP address, cookies e.t.c But the ways I show help make it look like it’s more a human that selenium

  • @shakils1921
    @shakils1921 Жыл бұрын

    it just open and close instantly.

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    Can you share code?

  • @shakils1921

    @shakils1921

    Жыл бұрын

    @@MichaelKitas thank you for your reply sir, It's now ok after updating chrome into 106 version.

  • @activeimmat5059

    @activeimmat5059

    Жыл бұрын

    @@shakils1921 Hey, i have the same issue, how do you uptade chrome to a lower version ?

  • @shakils1921

    @shakils1921

    Жыл бұрын

    @@activeimmat5059 that was not lower that time when I commented. But I recommend you to use chrome portable for selenium. Because it’s not update and always same chromedriver.

  • @rubigarcia431
    @rubigarcia4314 ай бұрын

    dejo de funcionar el dia de ayer

  • @nerffguy

    @nerffguy

    2 ай бұрын

    el que?

  • @MichaelKitas

    @MichaelKitas

    Ай бұрын

    ¿Dónde dejó de funcionar?

  • @phuoctainguyenlang5668
    @phuoctainguyenlang56689 ай бұрын

    C#

  • @Enzo_Barletta
    @Enzo_Barletta28 күн бұрын

    What is that IP and port you used for proxy? "106.122.8.54:3128"

  • @pduyet97
    @pduyet97 Жыл бұрын

    how can I bypass cloudflare in selenium java :((((

  • @MichaelKitas

    @MichaelKitas

    Жыл бұрын

    I haven't used java, either any cloudflare experience sorry

  • @abdoahmed1231
    @abdoahmed12312 жыл бұрын

    I have a question please When I use a proxy server and I follow people on Instagram, the follow-up is not done, for example, I have 5 followers on Instagram and I request to follow 100 people, that account is not done, they are still following 5 people, I just want to know how this happens, can the effect of the reverse proxy or forward proxy or requests sent can be redirected please help me

  • @MichaelKitas

    @MichaelKitas

    2 жыл бұрын

    Instagram blocks your account from following other users, like e.t.c when they see you have been spamming that action. Also, I am not sure how you are requesting for the follow, try following just 1 and see if you have that issue. You probably need to put a timeout between each follow so instagram doesn’t block you. In that case changing ip address doesn’t matter

  • @user-ge5ek5wi7j
    @user-ge5ek5wi7j11 ай бұрын

    hi bro i need to talk to you in a bot project plz tell me how to contact you

  • @MichaelKitas

    @MichaelKitas

    11 ай бұрын

    Please reach out to me at support@websidev.com with the project details and your budget range.