Data Migration From One Site to Another using TaskMagic
Building a Web Scraping Automation for Horse Details with Task Magic
This video demonstrates how to create an automation for scraping horse details from a website using Task Magic. It involves exporting horse names and links to a Google Sheet, setting up automation steps to scrape additional details like name, price, and specific links, and looping over pages of listings to gather comprehensive data. The process includes filtering the search to thoroughbreds, using a list trigger to handle pagination, and setting up a loop that avoids reloading the URL for every scrape. The final step involves setting up a second automation that loops over URLs from the Google Sheet to gather more detailed information about each horse and exporting this data to another Google Sheet. This tutorial showcases the setup of triggers, the scraping of lists and single items, and the use of variables and Google Sheets to manage and store the scraped data efficiently.
00:00 Introduction to Web Scraping Automation
00:00 Setting Up the Initial Scraping Automation
01:17 Navigating and Scraping the Website
03:05 Exporting Data to Google Sheets
03:41 Adjusting the Automation Loop
05:12 Finalizing the Scraping Process
07:22 Setting Up the Second Automation for Detailed Scraping
09:48 Configuring the Loop and Export for Detailed Data
15:01 Running the Detailed Scraping Automation
16:32 Conclusion and Review of the Automation Process
LINK TO SCHEDULE A BUILD SESSION: calendly.com/taskmagickyle/au...
Comment below: What are you looking to automate today?
SUBSCRIBE to this channel: kzread.info?...
Tell us in the comments what questions you'd like answered, we'll mention you on the show!
Follow Me:
TikTok: / thejeremyredman
Twitter: / thejeremyredman
Instagram: / thejeremyredman
Make.Taskmagic.com/
Пікірлер: 16
Thanks
@TheJeremyRedman
2 ай бұрын
Thanks to YOU!! I bleed for the engagement
Great one, thank you, Kyle! Can an additional column be used on the sheet as a conditional trigger? Let's say column "scrape" if = 1 scrape; after having scraped the page, automatically change it to 0
@TheJeremyRedman
2 ай бұрын
Sounds like you could use find row / update row actions for that
Great demo Kyle! Quick question about scraping : let's imagine I want to schedule every week the scraping of the horses. I could be unaware of the number of pages / horses, because the listing could be updated daily, weekly,.. Is there a way to say "I don't know how many items to scrape, so go ahead until you reach the last page" ? I didn't use TaskMagic a lot for scraping yet, but that's coming and I will need to schedule daily scraping :) Anyway, great demo of TaskMagic power! ❤
@TheJeremyRedman
2 ай бұрын
You’re advanced Man! Lol. We’re making a tutorial on this it’s going to be like a 15 minute tutorial it’s kinda hard. Stay tuned
@chamerlik
2 ай бұрын
@@TheJeremyRedman wow, cool as that means there's already a way to do that :)
My thoughts: You can add a loop within the loop to get this done in a single automation. (I've tried it for another use case. (I will share it in the coming days)
@TheJeremyRedman
2 ай бұрын
Yeeeees!!! Share that inside the community my man!!
@kyle.builds
2 ай бұрын
Yes you could!! Breaking it up like this is a really easy / approachable way before users can use the loop action. Looking forward to seeing your examples!
Didn't watch so far but I am sure it agai is a great demo, as always. PS: Ceterum censeo Zapier delendam esse ;-)
@TheJeremyRedman
2 ай бұрын
I’m very appreciative of this!! As you know
@udo9999
2 ай бұрын
@@TheJeremyRedman 👍
Hey lazy tribe! The automation window (showing the steps) vanishes "consistently" it's completely unstable (desktop). And, no it's not hiding behind my other windows, it's actually unreachable.
@TheJeremyRedman
2 ай бұрын
Haha lazy tribe. I’ve been calling us “lazies” lol. Lemme think
@TheJeremyRedman
2 ай бұрын
Is this JUST for desktop apps? Like when you’re automating desktop apps?