As long as you're not hacking into a database or something, scraping is perfectly legal because all you're doing is gathering data that is being made publicly available to visitors on their websites

@NeuralNine

2 жыл бұрын

Yeah but if you are constantly bombarding a website with requests to get "live data" it can be really annoying.

@CrazyFanaticMan

2 жыл бұрын

@@NeuralNine I agree, I can't remember which site it was or what I was scraping, but I definitely sent way too many requests to the site and eventually it just didnt want to give me access to its pages anymore haha so your video is extremely useful

@starling0216

2 жыл бұрын

@@NeuralNine proxy selenium python

@charlesgormley9075

2 жыл бұрын

It’s not illegal but could go against the websites terms of service which if ignored could result in a legal case, if they pursue legal action.

@samuelcooper9880

2 жыл бұрын

Lots of devs layout the rules anyway on the site. Many sites just straight up give out api keys to help.

LOL of all the books to pick first...Golden Condom. hahaha. you dealt with it quickly. good on you, funny shit.

Does it automatic rotating the proxies or we have to change the country with code for each time that our ip is blocked?

Do you know how to scrape data bypassing the CloudFlare? Been trying for weeks now

can we use it with selenium ? And what about csrf problem for example doing instagram scraping ?

Thanks. It is possible with asynic?

Bester Tutorial wie immer danke !!!

does this solution allow to pass 'headers' info if calling an api ?

How can I get terminal output stored into my.txt file

Could you do an example with Playwright and Oxylabs?

Secondly, if you went to Settings on PyCharm, and set the Proxy manually to OxyLabs, would that bypass the proxy code in your video?

Is multithreading/parallel webscraping also okay even with proxy?

What would be the code for scraping proxies for SB

How does portfolio for web scrapers look like?

Can you please do a video where you automatically download multiple files from a website using python. It can be pdf or csv files.

why am I receiving the message after a print(response.json()) ' {'message': 'provided url is not supported'}' . This happens when I try to web scrap from google scholar. Does this mean that the OxyLabs IP's didn't work? Because it couldn't support the url from scholar?

My question is, isn't it fair to conclude that some some web servers are just so adamant about not getting scraped, and good at implementing it, that no amount of coding is ever gonna get past their security? With one particular website that I need data from, which already has my IP blocked, I can get to it's home page, which just has a disclaimer, with a VPN connection, but as soon as I get to the page where I need to fill out the search form, I get nabbed and the IP from the VPN turns out to be listed as suspect and from that point, blocked. I would like to think that there are tricks in my court that I could use but I'm not seeing it at all. Can you shred some light on this? I'd rather know that it's not possible and stop pulling my hair out, than think I just need to do a little more, when that's not the case. Thank you

You did great job, 👍

Greetings from brazil. Nice video!

What to do if this error occurs ? response_html = response.json()['results'][0]['content'] KeyError: 'results'

I remember that book, A Light in the Attic. Hehe

Why do you use statistics.mean(prices) instead of just avg(prices)? Thanks in advance

@midmanphilippines

2 жыл бұрын

Speed

Thanks

just stumbled upon this vid about web scraping with proxies in Python. Been thinking of giving it a shot, especially with Proxy-Store's proxies. Anyone got experience with this? Lemme know!

Dude why are you so good??

@NeuralNine

2 жыл бұрын

Glad you like the video! :)

@billigerfusel

2 жыл бұрын

Practice and big brain

Proxy selenium python videos

nice!

🔥🔥🔥

@NeuralNine

2 жыл бұрын

🙏🏻

I wish I was smart enough to understand this video lol.

@13:37 couldn’t you just do requests.post

@NeuralNine

2 жыл бұрын

Yes I think that would have the same effect

Hi