How To Use ChatGPT To Fully Automate Web Scraping

Ғылым және технология

Web scraping is the process of automatically extracting data from websites using scripts. ChatGPT is able to generate the web scraping script code for you.
The example of this tutorial is showing that ChatGPT is a great help when generating script for web scraping. In fact, we were able to just input our simple requirements into ChatGPT and as a result we received a Python script which is able to perform the task without the need of any adaptions. That is making web scraping a lot easier and gets you started in no time.
Blog Post: www.codingthesmartway.com/how...
This is a CodingTheSmartWay tutorial.
---------------------
Musik:
Night Owl by Broke For Free is licensed under a Creative Commons Attribution License.
(freemusicarchive.org/music/Bro...)
#chatgpt #webscraping #python #ai #openai #artificialintelligence #webdevelopment

Пікірлер: 57

  • @alkuzad
    @alkuzad Жыл бұрын

    For me, it did not show the potential of the tool. You set requirements and then instead of fixing it with ChatGPT you just removed it. Prompt has to be more descriptive and not just "parse the website" to get good results. Pip install command is incorrect but because you had it already pre-installed it works.

  • @gaa535g
    @gaa535g10 ай бұрын

    Thank you! That was easy ❤

  • @mangeshsawant6361
    @mangeshsawant6361 Жыл бұрын

    helpful video, thank u

  • @hamburger--fries
    @hamburger--fries6 ай бұрын

    This was super helpful - Like a gateway drug. I used it a little differently. I scrapped 2000 books from my companies site and then removed various html and other odd things that occured in a OCR scan years ago. Anyway thanks!!!!

  • @mikecorbean8819
    @mikecorbean8819 Жыл бұрын

    Well, my CGPT buddy has been troubleshooting this w/ me now for about an hour. I'm on my 4th version of code from my partner. All 4 run successfully but the resulting csv file only contains the headers: column A: "Movie" column B: "Title" Now she's suggesting we attack this one using the API - argh!

  • @yaakovk9468
    @yaakovk94683 ай бұрын

    it's can work with linkedin to or google search ? for emails or more ?

  • @ahmadmerei7979
    @ahmadmerei7979 Жыл бұрын

    Thank you for sharing but still want more details on how I can learn this step by step? any reference, courses?

  • @xsw882

    @xsw882

    Жыл бұрын

    chatgpt will explain it to you step by step

  • @user-wl7iu4ec2c
    @user-wl7iu4ec2c Жыл бұрын

    What if BeautifulSoup and openai prompt to be used to analyse the data and fetch the information what is required. any one has tried out.

  • @comptegmail273
    @comptegmail273 Жыл бұрын

    do you know how to scrape data frmo glassdoor

  • @ryandunner
    @ryandunner10 ай бұрын

    Hey! I am looking to scrape data from a website, specifically news articles from a few sources I have. I want to scrape this data and then have it automatically plugged back into chat gpt for further prompting, is that possible?

  • @asifmian43

    @asifmian43

    2 ай бұрын

    yes, you can contact me

  • @bigd6566
    @bigd6566 Жыл бұрын

    Hello, can you please show me how to get just the actors from all the 250 movies without including the directors? Thanks

  • @user-ch8hh5oy6g
    @user-ch8hh5oy6g11 ай бұрын

    where do I put the code, pleas? I am not IT at all, BUT willing to learn this scrape skills as I considr it great stuff for us sales people. P.S. how would you suggest to scrape 1. all 5* hotels in New York and 2. car rentals in all Caribbean? Muchas gracias, joseph

  • @viljevisser1442

    @viljevisser1442

    8 ай бұрын

    Hi! I work with a lot of sales teams to generate new cold leads for them via scraping. Let me know if I can assist.

  • @adied7725
    @adied7725 Жыл бұрын

    Hi thanks for the video, How to visualise this data in wordpress?, please I'm trying to build a we scraper or url parser... Any ideas much appreciated thanks

  • @bifrostbeberast3246

    @bifrostbeberast3246

    Жыл бұрын

    ChatGPT cann explain it to you

  • @adied7725

    @adied7725

    Жыл бұрын

    @@bifrostbeberast3246 how?, please i'm not a develper any pointers?

  • @Peepstep
    @Peepstep8 ай бұрын

    How to about getting contacts from sites like jist dial india mart..make a video about getting contacts from websites use chat gpt

  • @jessbailie1628
    @jessbailie1628 Жыл бұрын

    can this be used to scrape email addresses? the addresses are publicly available on the company website.

  • @jaypg3d977

    @jaypg3d977

    Жыл бұрын

    you can scrape anything as long as there is no restrictions that prevent it

  • @justaguy905

    @justaguy905

    Жыл бұрын

    put a requirement of '@' to scrape it

  • @ilyasalmas8841

    @ilyasalmas8841

    Жыл бұрын

    @@justaguy905 ?? explain

  • @dr.samuelburbanks4905
    @dr.samuelburbanks4905 Жыл бұрын

    How do you get to the terminal. so many steps left out.

  • @MichaelShingo

    @MichaelShingo

    Жыл бұрын

    Windows: search "Command Promp", Mac: search "Terminal"

  • @vasilepelmus3308

    @vasilepelmus3308

    Жыл бұрын

    Exactly! But they said how to create an account with Chat GBT for 2 minutes🤦‍♂🤦‍♂🤦‍♂

  • @abhishekprakash9803
    @abhishekprakash9803 Жыл бұрын

    i have a one questions like ...........we want to build a chatbot .....for confidential document ....can chatgpt give the answers from my documents how to link my document to chatgpt.

  • @MirGlobalAcademy
    @MirGlobalAcademy3 ай бұрын

    .empty what is the purpose of it?

  • @TheCentoru
    @TheCentoru7 ай бұрын

    Sorry mate, but the error fix at 8:15 makes this approach as NOT fully automated

  • @tapforall
    @tapforall11 ай бұрын

    doesn't work

  • @SkillSage_Pathways
    @SkillSage_Pathways6 ай бұрын

    I am feeling dizzy after this video. Still too complicated and unaccessible to the layman.

  • @InnerFlame8

    @InnerFlame8

    4 ай бұрын

    That’s the easiest video I have watched today

  • @bifrostbeberast3246
    @bifrostbeberast3246 Жыл бұрын

    People asking stupid questions in the comments without realizing that they get an instant answer if they would ask ChatGPT, which is the very topic of the video. MAybe it's better those people don't know how to do the stuff they ask for. It's like giving a monkey a gun.

  • @akshayy_xd

    @akshayy_xd

    Жыл бұрын

    Hahahahaha

  • @Cahangir

    @Cahangir

    9 ай бұрын

    dont worry, they'll learn when they are banned from a certain website ))

  • @dr.samuelburbanks4905
    @dr.samuelburbanks4905 Жыл бұрын

    You are skipping important steps like what is the app that you used to paste the code in, yes you said it but I didn't understand what you said.

  • @MichaelShingo

    @MichaelShingo

    Жыл бұрын

    VS Code

  • @bifrostbeberast3246

    @bifrostbeberast3246

    Жыл бұрын

    @@MichaelShingo The video shows a man who asks ChatGPT to program for him and it does. You might get the idea that you also could ask ChatGPT all the things you want to know. Like: "What is a software called, where programmers program inside?"

  • @heroe1486

    @heroe1486

    3 ай бұрын

    @@bifrostbeberast3246 And chatGPT would figure out if this man was using Neovim, vim, Sublime Text, Visual Studio, Vscode, Atom or whatever ? This video is just some clickbait that made some people think it was really automated while he copy pasted some basic code you'd figure out by reading some README for 5 minutes, ofc it'd bring clueless non programmers.

  • @MarblelessMedia
    @MarblelessMedia7 ай бұрын

    This is no longer working. I'm told it goes against internet policy.

  • @shivamkk007
    @shivamkk0075 ай бұрын

    Basically Taught how to use chatGPT... That wasn't what I came here for

  • @v1s1v
    @v1s1v10 ай бұрын

    This actually shows that it's not that easy to fully automate web scraping with ChatGPT. There are AI-powered web data extraction providers like Kadoa that actually make it easy.

  • @tajinetopf95
    @tajinetopf95 Жыл бұрын

    danke du bist im deutschen netz

  • @bifrostbeberast3246

    @bifrostbeberast3246

    Жыл бұрын

    Das "Th" hats verraten :D

  • @creamsodabish
    @creamsodabish Жыл бұрын

    yeah man, you skipped over a LOT of stuff. all good. i'll figure it out

  • @bifrostbeberast3246

    @bifrostbeberast3246

    Жыл бұрын

    He literally showed in the video how you can ask ChatGPT everything you want to know and it will even code for you. But yeah, I am sure you will figure it out.

  • @kakamoora7874
    @kakamoora787411 ай бұрын

    Jokes 😂

  • @kirill_good_job
    @kirill_good_job8 ай бұрын

    your experiment is not replicate

  • @nadranaj
    @nadranaj10 ай бұрын

    misleading title. It should have been "How to use ChatGPT to generate code to do web scraping"

  • @heroe1486

    @heroe1486

    3 ай бұрын

    Yeah this video doesn't show any "automated" webscrapping, he's just asking chatGPT for basic code you'd figure out by reading some README, some library genuinely do some sort of scrapping automation, requests and Beautiful soup are basic tools that don't do that and would actually be used in the core of such projects. It's a video for laymen I guess.

  • @brandonjoseph3422
    @brandonjoseph3422 Жыл бұрын

    Nonsense

  • @ibessf
    @ibessf10 ай бұрын

    Your Englisch Sound a bit German 😅

  • @MichealS-bi8xt
    @MichealS-bi8xt Жыл бұрын

    ChatGPT is just used as a scaffolding tool there. Nothing revolutionary. Sure it will make you gain a lot of a time if you already know how to code. If you don't know how to code AT ALL, you will struggle, even with Chat, you will probably struggle LESS than without Chat but don't expect to produce top notch apps. This is sold by people who just want YT views and sell tutorials.

  • @bbrrrrr6553
    @bbrrrrr6553 Жыл бұрын

    ChatGPT is just used as a scaffolding tool there. Nothing revolutionary. Sure it will make you gain a lot of a time if you already know how to code. If you don't know how to code AT ALL, you will struggle, even with Chat, you will probably struggle LESS than without Chat but don't expect to produce top notch apps. This is sold by people who just want YT views and sell tutorials.

Келесі