Automate File Handling With Python & AWS S3 | Five Minute Python Scripts

Ғылым және технология

Пікірлер: 110

  • @breadenthusiast
    @breadenthusiast3 жыл бұрын

    Your awesome Derrick! Some more videos on AWS and APIs would be wonderful! Keep up the great work!

  • @petermcentee9922
    @petermcentee99223 жыл бұрын

    This is exactly what I needed for work, thanks so much!

  • @lancehoward1609
    @lancehoward16094 жыл бұрын

    Great job. Finally that can explain step by step what you are doing!

  • @abhinavkannojia1053
    @abhinavkannojia10534 жыл бұрын

    Big Fan of these pythonic tricks..Nice work there. Please keep it coming !!

  • @unitech23
    @unitech237 ай бұрын

    Amazing Derrick. Great Job Dude!!

  • @vivekr4690
    @vivekr46904 жыл бұрын

    Keep uploading such useful videos! Thanks! I learnt some new concepts here!

  • @deepquest

    @deepquest

    2 жыл бұрын

    All about S3 Bucket; Subscribe for more such videos. kzread.info/dash/bejne/np6o1LSQZpzgYZM.html

  • @Anwar-Mian
    @Anwar-Mian2 жыл бұрын

    Excellent, Derrick!!!

  • @Sagar_Tachtode_777
    @Sagar_Tachtode_7773 жыл бұрын

    Awsome Job. Thank you Derrick...

  • @oldguywholifts
    @oldguywholifts3 жыл бұрын

    Has to be said.. direct and precisely what the title says. EXCELLENT!

  • @RenandOliveira
    @RenandOliveira Жыл бұрын

    Great Video Derrick! Go Vols!

  • @just5two163
    @just5two1632 жыл бұрын

    Thank you Derrick. Nice tutorial. Thanks for your time! It helped me a lot.

  • @Reptar4000
    @Reptar40003 жыл бұрын

    You’re the man. Need more content!

  • @HopeUnveiled
    @HopeUnveiled2 жыл бұрын

    Wish i was this beautiful hahah Great vid dude! Helped me a lot.

  • @colorfuljosh
    @colorfuljosh Жыл бұрын

    Great video, thanks from Australia dude

  • @vijaypingale7
    @vijaypingale73 жыл бұрын

    Thanks man ! Great video!

  • @fbzz64
    @fbzz643 жыл бұрын

    You explain very well, thanks for sharing your knowledge .. Greetings from Argentina!

  • @renyildiz5155
    @renyildiz51552 жыл бұрын

    I am fairly new to data but you are awesome. Thank you!!

  • @DonellGrace
    @DonellGrace4 жыл бұрын

    Amazing. Well done

  • @kunaljha6204
    @kunaljha62042 жыл бұрын

    Stupendous tutorial....Need more such tutorials

  • @amnksngh29
    @amnksngh292 жыл бұрын

    Thanks Derrick!

  • @pepex90
    @pepex902 жыл бұрын

    Thank Derrick !!

  • @bismarckandu6069
    @bismarckandu60692 жыл бұрын

    This is natural teaching.

  • @chadripley2776
    @chadripley27763 жыл бұрын

    great video. One issue I had was you have to set the region in the arguements ofr creating the client object if it isnt configured already.

  • @AbhishekBhuva
    @AbhishekBhuva4 жыл бұрын

    I am a Vlogger but as a professional, i am a software engineer too. I think your video is more interactive since its 6-10 min.

  • @anilct6749
    @anilct67493 жыл бұрын

    Thanks man. Much appreciated!

  • @deepquest

    @deepquest

    2 жыл бұрын

    All about S3 Bucket; Subscribe for more such videos. kzread.info/dash/bejne/np6o1LSQZpzgYZM.html

  • @nowyouknow2249
    @nowyouknow22494 жыл бұрын

    Well done bro. Please more vids on AWS.

  • @deepquest

    @deepquest

    2 жыл бұрын

    All about S3 Bucket; Subscribe for more such videos. kzread.info/dash/bejne/np6o1LSQZpzgYZM.html

  • @3h936
    @3h9362 жыл бұрын

    Thank you, it's amazing to video !!!

  • @adrianmisak07
    @adrianmisak07 Жыл бұрын

    thank you, this was great

  • @keithsmalling9723
    @keithsmalling97232 жыл бұрын

    Brilliant tutorial well explained

  • @AdityaBolt
    @AdityaBolt4 жыл бұрын

    loved the video..

  • @yaseenimuhammadraja9461
    @yaseenimuhammadraja94613 жыл бұрын

    Great video 👍

  • @DanielArturoSanchezDeoleo
    @DanielArturoSanchezDeoleo2 жыл бұрын

    thank you greetings from the Dominican Republic

  • @GG1440
    @GG14403 жыл бұрын

    This is very helpful, thank you.

  • @deepquest

    @deepquest

    2 жыл бұрын

    All about S3 Bucket; Subscribe for more such videos. kzread.info/dash/bejne/np6o1LSQZpzgYZM.html

  • @paulschrum9222
    @paulschrum92223 жыл бұрын

    3:36 -- file is a python keyword. Don't use Python keywords for variable names. Use something like a_file or my_file.

  • @stahmedpendragonguinevere2711
    @stahmedpendragonguinevere27114 жыл бұрын

    Keep up the good stuff matey!

  • @rajkishan3690
    @rajkishan36903 жыл бұрын

    Well Explained.

  • @vamshidharr3434
    @vamshidharr3434 Жыл бұрын

    Awesome !!!

  • @amatulathok597
    @amatulathok597 Жыл бұрын

    Thank's a lot dude

  • @MuhammadAli-zg2nw
    @MuhammadAli-zg2nw3 жыл бұрын

    you deserve the like and sub

  • @AshisRaj
    @AshisRaj Жыл бұрын

    Excellent

  • @japeshgawale9640
    @japeshgawale96403 жыл бұрын

    You are awesome Derrick! Could you Please upload a video of automating data transfer from AWS S3 to AWS Redshift (Using AWS Glue) or any other appropriate method .

  • @BehruzbekOtayev
    @BehruzbekOtayev3 жыл бұрын

    Thank you

  • @lucasneves3465
    @lucasneves34652 жыл бұрын

    amazing, it was very useful I have a question: how can I send the file to a specif path in my bucket?

  • @lotusmojo
    @lotusmojo3 жыл бұрын

    Awesome

  • @erich.3247
    @erich.32473 жыл бұрын

    Thanks for the vid man! Noob questions here. Can the S3 subdirectory be created on the fly from within the python script? Say I get a .py file but I have no python subdirectory in my bucket. If my script calls for storing the file in my bucket's non-existent "python" subdirectory will it be created or do I have to manually do it beforehand? Thanks!

  • @velnathravanan1765
    @velnathravanan17652 жыл бұрын

    In the Python file, write a program to get all the files from a public S3 bucket named in random list. In there there might be multiple files, but your program should find the file with the prefix __cb__, and then output the full name of the file. You should use the boto3 module to solve this challenge.

  • @gabopython
    @gabopython Жыл бұрын

    good video

  • @willd0g
    @willd0g3 жыл бұрын

    That’s great

  • @granand
    @granand3 жыл бұрын

    Thank you so much. Derrick Please can you take me along with you in a journey to use Python for data manipulation using strings, lists, tables, SQL and excel with regex.

  • @ncTechlearning
    @ncTechlearning4 жыл бұрын

    Could you please make a video on file transfer from sftp to amazon s3 using python

  • @aajadsingh8050
    @aajadsingh80504 жыл бұрын

    Nice

  • @BekaBelle
    @BekaBelle Жыл бұрын

    Thanks Derrick! What if i wanted to upload csv files one each day and to also check if the file is already in s3 then dont upload?

  • @ShivamGupta-dj3eh
    @ShivamGupta-dj3eh4 жыл бұрын

    It's not working for me. Showing error: boto3.exceptions.S3UploadFailedError: Failed to upload upload_data_S3.py to new-bucket/python/upload_data_S3.py: An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist

  • @rupeshbhuju2897
    @rupeshbhuju28972 жыл бұрын

    Hi there! If you could suggest some python approach to write a file into multiple target buckets from source bucket, that would be helpful. Thanks

  • @xin6129
    @xin61293 жыл бұрын

    This is great! Thanks for sharing! Could you also do a FTP one? I am very new to python/

  • @DarkStrado

    @DarkStrado

    2 жыл бұрын

    You should never use FTP anymore. SCP is more secure.

  • @raja4usekhar
    @raja4usekhar3 жыл бұрын

    Can you do /suggest best automation tools to work with aws

  • @sathishsuganya2013
    @sathishsuganya20132 жыл бұрын

    If i wanted to add some file in to the specific folder (Existing on S3 Bucket)how i do that?

  • @dreambig9794
    @dreambig97943 жыл бұрын

    I have my bucket like "parent_Bucket/Chield_Bucket", Now I want to upload my file to Chield_Bucket. Could you please suggest what I need to do for this.

  • @islauddin
    @islauddin2 жыл бұрын

    Looking for a way to transfer files from our Sharepoint location to a AWS S3 bucket using Python. Any good suggestions on how to go about doing it?

  • @hesamulhaque2197
    @hesamulhaque21972 жыл бұрын

    Great video like always! Can you also make a video on cron job?

  • @deepquest

    @deepquest

    2 жыл бұрын

    All about S3 Bucket; Subscribe for more such videos. kzread.info/dash/bejne/np6o1LSQZpzgYZM.html

  • @mwredfern
    @mwredfern2 жыл бұрын

    just reviewed another video from someone that literally walked through this same scenario. however, their video was 57 mins! no. i did not watch the 57 min video. i watched about 3 mins and saw your video sitting in the sidebar screaming '7 minute video over here!' haha. to the point and just the facts. no showboating knowledge and skills. if im ever drowning, i hope you are on the shore to save me and not the person who makes a 57 min video on 'how to jump in and save a drowning person'. haha

  • @surajiyer7406
    @surajiyer74063 жыл бұрын

    If we need to read multiple files from AWS S3 bucket, what must be the code in Python?

  • @harisantho
    @harisantho Жыл бұрын

    Hi.. thanks for this video How to copy one file from ec2 instance particular path to s3 bucket

  • @withkonan
    @withkonan Жыл бұрын

    how can i upload a 1 gb video to s3 bucket quickly . or if i could upload it asynchronosly(i.e., my page should get redirected to next page and the video can get uploaded at background)

  • @romachd01
    @romachd013 жыл бұрын

    @Derrick Sherrill everytime I try to run the code it picks up files from windows32 directory. I have the .py file inside a folder with few test .txt files but it keeps picking up txt files from the win32 dir. Whats happening, please help

  • @AnAyahaDay
    @AnAyahaDay3 жыл бұрын

    How can we delete all files from a S3 folder? ( not the folder, but all files in a folder )

  • @celsoalbertoreyesugalde4792
    @celsoalbertoreyesugalde47923 жыл бұрын

    Excellent video!!, could you do something similar but uploading automatically to Google Cloud Platform?

  • @mainakbiswas6541
    @mainakbiswas65412 жыл бұрын

    Nice video. I have one question. I don't want to download the content from S3, and I want to know the content details like no. Of .txt, no. Of. py within that bucket. I am using aws s3 ls bucket_name/ - - recursive - - human-readable - - summarize It shows the total size and objects no. But can't find the file counts with respect to their extensions.

  • @alejandroaparicioguerra1818
    @alejandroaparicioguerra18182 жыл бұрын

    Good sample could you please make an example of how to send email with lambda too

  • @rahulmeena79
    @rahulmeena796 ай бұрын

    I uploaded files to S3 yeahhhhhhh

  • @senthilcm
    @senthilcm3 жыл бұрын

    Hi How can read the files browser for S3

  • @jamilmjq
    @jamilmjq Жыл бұрын

    need idea how to copy files to s3 bucket in UC4

  • @lucasrosa7781
    @lucasrosa77812 жыл бұрын

    Hi Derrick!! I want to know with its possible to do this with a AWS Lambda for automation pourposes?? (sorry about the english, im from Brazil)

  • @soniathomase7757
    @soniathomase7757 Жыл бұрын

    Do you have any idea how i can access files in a s3 bucket with just my IAM role? I dont have the access keys. Is this possible?

  • @parag4982
    @parag49823 жыл бұрын

    what to do when the s3 has multiple buckets and we need to choose a particular bucket??

  • @un_known7299
    @un_known72992 жыл бұрын

    How to securely insert the Access key and password to the python or c# Because using your method any one can decompile my program and have my access key and secret key ?

  • @priyankapatel9461
    @priyankapatel94612 жыл бұрын

    How can I Write a Python code to create AMI and send email to all for AMI creation. also create spot instance on based of backup AMI?

  • @philippephilippe1265
    @philippephilippe12653 жыл бұрын

    Hi Derrick and all I tried this method and I am getting the error below ImportError: cannot import name 'access_key' from 'secrets' (C:\Program Files\Python37\lib\secrets.py) did anyone had same error? and how do you go about it?

  • @MichaelDavis-hk8vl
    @MichaelDavis-hk8vl3 жыл бұрын

    Can I do this from Juypter Notebook?

  • @omkarbabar4420
    @omkarbabar44203 жыл бұрын

    Hi.. Can you please tell how to do same thing on heroku platform?

  • @rishvish
    @rishvish2 жыл бұрын

    Can we download files from AWS in any similar way?

  • @sarahkim3856
    @sarahkim38563 жыл бұрын

    Hi, I was trying this and I'm getting an error of "FileNotFoundError: [WinError 2] The system cannot find the file specified:" I tried searching it everywhere but I couldn't figure it out. Any advice?

  • @carlbjorklund2703
    @carlbjorklund27033 жыл бұрын

    Hi, thanks for great video. I did though run into an issue if you have time. I followed the tutorial and I can get the file (image.jpg) from my images directory (I see this image by running print("file", file) in the os.listdir for-loop) but when I run: "client.upload_file(file, upload_file_bucket, upload_file_key)" I get error "FileNotFoundError; [WinError 2] Can't find the file 'image.jpg' Do you have an idea what the issue can be? Thanks

  • @tahmeedhossain2910

    @tahmeedhossain2910

    3 жыл бұрын

    Did you find any solution for this ? I am having the same issue.

  • @tahep3906

    @tahep3906

    2 жыл бұрын

    same with me, any solution?

  • @rickramirez94
    @rickramirez943 жыл бұрын

    Do a aws series !

  • @wilianuhlmann5284
    @wilianuhlmann52843 жыл бұрын

    how to copy file in other directory?

  • @ajtam05
    @ajtam052 жыл бұрын

    Are you able to download, as well?

  • @rattaponinsawangwong5482
    @rattaponinsawangwong54823 жыл бұрын

    I wonder if you could do the same with google drive? Google drive is more popular in my region.

  • @preyasprathap
    @preyasprathap4 жыл бұрын

    Nice video.. i am thinking of directly deleting the files after it gets uploaded.

  • @CodeWithDerrick

    @CodeWithDerrick

    4 жыл бұрын

    Thanks! You would just add this inside your for loop after the upload statement: os.remove(file)

  • @mayukhchattaraj5552
    @mayukhchattaraj55523 жыл бұрын

    Please upload to a code for S3 bucket to S3 bucket transfer of files.

  • @mwredfern
    @mwredfern2 жыл бұрын

    if you have a video for SFTP, please share link. :)

  • @mukeshk2553
    @mukeshk2553 Жыл бұрын

    Bucket to bucket automation how to do

  • @MrPawan412
    @MrPawan4122 жыл бұрын

    Awesome Derrick , I have one requirement where i need to upload the file from S3 to Sharepoint Site using python in AWS EC2. Any help please ???

  • @deepquest

    @deepquest

    2 жыл бұрын

    All about S3 Bucket; Subscribe for more such videos. kzread.info/dash/bejne/np6o1LSQZpzgYZM.html

  • @narendrasuryawanshi5815
    @narendrasuryawanshi58154 жыл бұрын

    Can system run a code even its shutdown. Lets say i don't work on Saturday & want to run a code. Will it possible?

  • @Byomakesh1

    @Byomakesh1

    3 жыл бұрын

    Narendra Suryawanshi For that you need to deploy your script on a server and schedule it

  • @harisantho
    @harisantho Жыл бұрын

    Hi 👋 thanks for your valuable KT. I'm requesting you to help me with a automated python script to check RDS snapshot name and snapshot created time for multiple Rds in multiple regions. Like N.V, ohio, Sydney, Singapore, Ireland, Paris, Belgium, etc..

  • @gregf9160
    @gregf91604 жыл бұрын

    More on AWS and Python would be great!

  • @XandoFootball
    @XandoFootball3 жыл бұрын

    Dude , Is there any reason why you aren't a Cloud developer for a company?

  • @ai.simplified..
    @ai.simplified..3 жыл бұрын

    I dont know what has happend that your chanel been unsubcripted

Келесі