Download my FREE ChatGPT Cheat Sheet: frankandrade.ck.page/08c94cf1c1

@ahmedmohammed-xo7rr

9 ай бұрын

It will be nice if we can apply this approach on Facebook posts and websites?

@yusufadeoye5600

5 ай бұрын

HI , please can we still assess the the free chatGPT cheat sheet. please do share again

Mind-blowing stuff! Wonder if combining it with Proxy-Store's proxies would level up the scraping game even more. Any ChatGPT fans here?

Great stuff @ThePyCoach! Question now is, how do we automate this scraping for multi-pages instead of manually saving each page as a new html file?

A terrific video that has been the cornerstone for our new team members. Thank you for sharing it!

Cool process. But what is the advantage of doing this process, instead of using pre-built scrapers already available like "Instant Data Scraper".

@xaviergonzalez6180

10 ай бұрын

You can tell Chat GPT code interpreter to do the same thing for any amazon page, just upload the html and run the same process. Instant data scraper is clunky and the interface looks old, I'm sick of clicking an dragging with all these data scrapping app extensions. Also certain websites don't work well with data scraper apps to many blockers. try extracting data from harbor freight with instant data scraper. its a nightmare! no thank you.

Very practical and helpful video with very detailed explanation!

Thanks, that's really helpful advice - and indeed pretty mind-blowing!

Hey PyCoach, learning a lot from you. Thanks man!

@ThePyCoach

Жыл бұрын

You're welcome!!

thank you!!! never thought about the save page as html and then uploading

is it same work as like other language website like Hindi , Japanese , Chinese and many more language ?

Thank you so much The PyCoach!

Top tier content bro. Easy sub.

You are the best, subscribed :)

Can you make a video like this that's scrapes the SEO keywords that a website is using?

@Daddytang67

Жыл бұрын

YES...LIKE AHREFS

Awesome!!!! and extremely useful!

how would you do it for the products page. So the tv plus the product pages details of the tv then so on.

Thank you very helpful.

This was pretty good, but I notice that none of these GPT methods for data extraction deal with pagination. How would we do this if we've got multiple pages of jobs on Glassdoor and you wanted to let chatGPT run through all of them?

@LucrativeSetbacks

7 ай бұрын

Thank you for pointing this out before I went too far in this video, as I am trying to scrape dynamic content loaded with JS

the interpreter code does not appear in the beta feautures to activate it just i have only plugin why? do you have a solution please

this is cool ! although It is probably not able to scrape any IMG/Links ?

can this be used to scrape multiple pages like 100 pages on a domain? i guess so, because one page would be silly

Thanks for the video! I tried both methods to scrape Udemy and it didn't work, since the content is in a container. Do you have a solution for this?

But it appears this only works with the paid version of chatGBT

So basically you made a tutorial about saving a file and loading it into chatgpt?

This is really cool❤

listening to this video gives me "Two Minute Papers" vibes. I was waiting for ""Alright everyone, hold onto your papers"

@tonycummings4588

6 ай бұрын

what a time to be alive

Try saving all the pages at once to reduce manual repetitive work

@robertoflores2078

5 ай бұрын

I guess the manual download part is intended for demonstration. You'd do this programmatically.

very nice! thumbs up.

I've had a huge increase in scrapers across my entire domain portfolio. Given the power of AI, I've managed to poison most of the scraped data and cause major headaches going forward.

@tryderrick

Жыл бұрын

What do you mean?

@bigpickles

Жыл бұрын

@@tryderrick upon detection, I feed a load of nonsense back in between legit data, and it totally ruins the dataset.

@bigpickles

Жыл бұрын

@@ronin4518 depends what you're scraping. Not my data

Hi, is there any way now to scrape all posts and comments of a subreddit? I need them for my research.

code interpreter isnt even an option was it removed? Maybe explain how to get it thatd be helpful

Why would I do this? What is the use case for this

But not the automation I am looking for - will keep up with the options choose and one click. Maybe a long way to complete it but will do everything I need. Really python and is with some chat chatbot content writing - might use ChatGPT - but the content never passes ai detection like clause on it's first time.

@pocketsfullofdynamite

Жыл бұрын

You really have to do the most part of the writing for SEO, based on the 'draft' or outline of the content.

Is it not possible to provide the url and then get it to loop through the pages till it gets to the last page and then get it to export the results to csv

@ThePyCoach

Жыл бұрын

If you pass the URL, ChatGPT will probably tell you that it's not able to perform web scraping or that is against the ToS

@anuroop345

Жыл бұрын

you can ask chatgpt for python script to scrap that page. and then you can run that script on your computer to loop and scrap.

@denwo1982

Жыл бұрын

@@ThePyCoach I would have thought someone would have made a plug-in for this now

@denwo1982

Жыл бұрын

@@anuroop345 I tried this but it’s not as easy, Chatgpt keeps having trouble with the looping side, plus a few websites require a login to access their site

we need to all this thing for scrap? And scrap for what?

Could have also just Did View Source, then copy paste, or maybe even use plugins with link reader and paste that viewed source URL in.

@egauci1

10 ай бұрын

View source won’t work if the page is rendered dynamically

@bobbynicholson5329

9 ай бұрын

true that@@egauci1

I love your contents especially this one.. Sir how can we offer this as a service. Who would need this service ? Can you help out maybe with a resources thank your

Thanks, but what if we ask it to use link directly with help of beautiful soup and selenium? What would happen

@ThePyCoach

Жыл бұрын

When I checked the code I could see it's using Beautiful Soup. I'm not quite sure what would happen if you specify to use Selenium. Just remember that code interpreter is mostly parsing the HTML file

If you used harpa or perplexity it will get the job done with no complications 😅

What is the purpose of doing this? What was the problem that makes your work its solution?

🎯 Key Takeaways for quick navigation: 00:00 🌐 Introduction to web scraping with ChatGPT Code Interpreter 01:07 📂 Saving a webpage as an HTML file 03:38 🧩 Extracting data from HTML using ChatGPT 04:08 📄 Exporting extracted data to CSV 05:03 🔁 Repeating the process for multiple pages 06:51 🌐 Scraping data from Glassdoor 08:13 🧩 Scraping specific data elements from Glassdoor 11:56 📄 Exporting Glassdoor data to CSV 12:37 🚀 Conclusion and call to action Made with HARPA AI

Hi Frank ty for the great walkthrough! I am trying to scrape google to deterniine companies near me that do not have a website. it appears to be a simple task to determine on the search results which ones to choose but for some reason gpt says the html structure of nesting elements is not allowing it to do so. Any ideas?

@lifistudyAnupam

11 ай бұрын

So have you done it?

@Corteum

9 ай бұрын

Just make a list of all the businesses near you. Then identify among those which could benefit from having a website. BOOMSHAKALAKA!

Hi, is Code Interpreter also available through the API?

@ThePyCoach

11 ай бұрын

Hi. I'm not quite sure

So how do I scrape data from the product pages as well?

@ThePyCoach

Жыл бұрын

Just upload its HTML file and follow the same steps

I don't wanna be that guy, but you can do this with a free tool called Harpa in a split of a second, it is nice to see the procedure but... 🤷‍♀

Did they remove the Code Interpreter plugin? How do you get it? I have a paid account. Thanks

@Fredsiika

Жыл бұрын

Try enabling it in your settings. Open Settings > Click on "Beta Features" > Press "Code interpreter" toggle button

Ok...but what I really need is for chatgpt to code a webapp like Ahrefs.

cool

I used it to scrap some websites. Last week my account was banned, said my activity was suspicions.

@ThePyCoach

Жыл бұрын

How could that happen? Did you upload too many HTML files?

@craftedbysrs

Жыл бұрын

​@@ThePyCoachNo i didn’t. I just uploaded direct link with scraper plugin on. And prompted the code interpreter to generate a new programming language that is more advanced.

@curious_one1156

Жыл бұрын

that is why you need to use langchain and make your own code agent. Also, only use interpreter to write code once, then handle it yourself.

@contentfreeGPT5-py6uv

Жыл бұрын

​@@curious_one1156how

@bigpickles

Жыл бұрын

Good. We don't like dodgy scrapers.

This scraping is very manual. There are much better ways.

@carkawalakhatulistiwa

Жыл бұрын

Next version of code interpreter

@izzyal-zyoud7563

Жыл бұрын

Could you please share them

@mishos.2228

Жыл бұрын

@@izzyal-zyoud7563 webpilot plugin, can scrape any number of pages with one prompt and by just providing the page 1 link

@klaaskay2685

Жыл бұрын

@@mishos.2228 Won't that get you banned anytime soon using just links?

@rancio3354

Жыл бұрын

Yea this aint web scraping. this is html extracting data, where is the web scraping at?? you are doing it manual…

this video should have been a short.

This was actually slower than classic python scraping.

@abiol1542

10 ай бұрын

Hey would you be able to assist in python scraping script and looping the scape for another search ?

If you know web scraping this is really nothing :D

What will happen if you want to scrape 1 million pages? 😀

@robertoflores2078

5 ай бұрын

1 million page downloads, and 1 million ChatGPT uploads.

This is more of a chatgpt-aided manual scraping. The processes aren't automated.

3:05 a empresa AVANCO SOCIAL tem o poder de transformar marcas e negócios por meio da habilidade em impulsionar redes sociais. É inspirador!

Dude 😂

I really don’t like this title. It feels dishonest. There’s are much better ways to use chatgpt to scrape

Watching your video made me angry

Not too useful. Stick to node puppeteer or perl/python command line scripts.

Bro how to train a own AI 😅 and make like chatgpt

@ThePyCoach

Жыл бұрын

I think you need to use Langchain for that

@dhuruvm

Жыл бұрын

@@ThePyCoach you can create a chatbot using Langchain how to make a chatgpt using Langchain

Please remove this video, you wasted my 15 minutes 😢

Clickbait 😅