To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/ThePyCoach . You’ll also get 20% off an annual premium subscription.

Thank you! I’ll try it out 👌🏻

Wow... Thats awesome! Thanks!

Thank you can you please create a video detailing how to update the GPTs system instructions you plan to use to account for the new @mentions feature for seamless teamwork and collaboration between your team of GPTs

Any plugin to overcome website that has security blocking ?

Just watching your videos made me realize how complicated im making my life. Such a brilliant way to scrape.

I've recently learn about ics. format file but I couldn't understand how to use it could you teach us how to use such file?

Youre a legend

Is there any method with open-source llms

Awesome!

The recommended scraper is unable to scrape pages requiring a log in and offers no way to log in prior to attempting a page scrape. It also has a relatively short timeout so slow responding or large pages can't be scraped.

@nitrogamesstudios5276

3 ай бұрын

And also these features could be work with paid Gpt's

Interesting. I have done some scraping before using Python to download date both in Excel and CSV format and insert to a MySQL database. In some of these examples I needed to use my username and password to access the data. Is this possible using Scraper?

is there a way to do that with Gemini?

Is there anything like these in hugging face or free as open source app? TIA.

hm sadly did not work as the scraper triggered anti-scraping policy of the websites I tried. This seems good for some minor quick scraping jobs but I wouldn't rely on it for actual production scraping.

@danlee1027

4 ай бұрын

Yes you may need to utilize proxies when scraping

Use gpt vision, more adoptive to html change

Can we do this without having the paid version of gpt?

Hi, I can't see the plugins option under model dropdown like you have @3:36 . I have a chat gpt plus subscription. I have also activated the plugins under settings.

@ThePyCoach

4 ай бұрын

Plugins have just been removed. You can either use the ScraperGPT or use an old chat where the Scraper plugin was activated (plugins still work in old chats)

I went to Perplexity as to get some data on web scraping. It used your YT channel as one of its references. Welcome to the AI big time brother 😂.

@luxipo9934

4 ай бұрын

The thing is that Perplexity sometimes is kind of “lazy” and that is due to the short token window it has. So for example if you ask for the updated market cap of 20 companies it will give you info that is not up to date for some of them.

Hi, do we have to give all the pages that we want to scrap ? Is it possible to give only the root page? Great video by the way :)

@ThePyCoach

4 ай бұрын

You'd need to give all the links, unless there's a pattern in the links that you can describe to ChatGPT

@bjraval9818

3 ай бұрын

​@@ThePyCoach So how do you scrape all business locations with their name and latitude and longitude values from Google Maps when it shows different number of business with each zoom levels and hides the rest at another zoom levels? Conceptually the problem is similar to providing ChatGPT with all page numbers but here we have to deal with different zoom levels. Any thoughts?

What about scraping data from private sites like LinkedIn? Is that also possible with a version of this method?

@ThePyCoach

4 ай бұрын

If it requires a login then it’ll be impossible with this method. You can try with my other method that involves the code interpreter and an HTML file (the video is on the channel)

is this paid version of chat gpt?I dont see " explore gpt"?

Awesome, but i rather manual way combine http header knowledge and linux, all command line

@jd2161

4 ай бұрын

Good. You do that

Sad part is most of the people don't have got 4 🙂

@fahimrafi9853

4 ай бұрын

Yeah 😢 Is there anything like these open source/ hugging face?

@jd2161

4 ай бұрын

Not sad

​ I added the very same comment to one of your replies below >>. @ThePyCoach So how do you scrape all business locations with their name and latitude and longitude values from Google Maps when it shows different number of business with each zoom levels and hides the rest at another zoom levels? Conceptually the problem is similar to providing ChatGPT with all page numbers but here we have to deal with different zoom levels. Any thoughts?

Thanks for sharing! when I try to download I get this error message "Failed to get upload status for /mnt/data/worood_product_list.csv" I tried different browser and nothing change. any thoughts on what could be the issue?