How I Would Learn Data Engineering 2024 (If I could start over)

Join Course Python for Data Engineering - datavidhya.com/courses
Learn how to become a data engineering expert in 2023 with this comprehensive guide. From understanding the basics to mastering advanced techniques, this video covers everything you need to know to excel in the field.
Whether you're just starting out or looking to refresh your skills, this tutorial is a must-watch for anyone interested in data engineering. Don't miss out on the opportunity to take your career to the next level - watch now and start your journey to becoming a top data engineer today!
Complete Data Engineering Roadmap 2023 - docs.google.com/document/d/1M...
Join Data With Darshil Discord Server: / discord
Timestamps
0:00 Introduction (very important)
2:28 Section 1 - Getting into Data Engineering
5:08 Section 2 - Building a strong foundation
10:14 Section 3 - Modern Data Stack and Advance skills
12:30 Learn Data Engineering + Projects
📷 Instagram - / darshilparmarr
🎯Twitter - / parmardarshil07
👦🏻 My Linkedin - / darshil-parmar
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
how to become data engineer in 2023, data engineering projects for free, path to learn data engineering, big data engineer for beginner, data science vs data engineer
✨ Hashtags ✨
#dataengineer #bigdata #2023

Пікірлер: 359

  • @DarshilParmar
    @DarshilParmar Жыл бұрын

    More projects are coming so make sure to hit LIKE button

  • @dhwanimehta4634

    @dhwanimehta4634

    Жыл бұрын

    Thank you Darshil your content has helped me a lot. I am very grateful to you.

  • @ilma300

    @ilma300

    Жыл бұрын

    Thanku brother. I want to become data engineer but I have no money for fee. Mostly people give m suggestions but I didn't understand. But now I'm understand. God bless u

  • @kuldeep4331

    @kuldeep4331

    Жыл бұрын

    Very helpful as always 😊 darshil. waiting for the projects

  • @shalalayusifli9734

    @shalalayusifli9734

    Жыл бұрын

    Thank you for great video! :) Very helpful

  • @rameezkhan4929

    @rameezkhan4929

    Жыл бұрын

    Hi thanks for the detailed roadmap. Will you please suggest and paid course for data engineering. I want switch into it from non it my age is 27 and i m doing BCA from second year

  • @baocao2492
    @baocao2492 Жыл бұрын

    For anyone looking for a summary: - Section 1: Build CS Foundation, SQL - Section 2: Dataware house, data processing frameworks, cloud platforms,

  • @guitarsz
    @guitarsz Жыл бұрын

    Oh my gosh Darshil! THANK YOU SO MUCH for the roadmap you posted. That is incredible and so generous. Thank you again for the great, detail and comprehensive video.

  • @Mariahemariaho
    @Mariahemariaho Жыл бұрын

    Great video, no irrelevant content or filler sentences. I have been studying on data engineering field for a while and I found that all information are very useful and including brief explanation about subjects. Thanks a lot for it.

  • @djgulston
    @djgulston Жыл бұрын

    Thank you, Darshil! This is very helpful. You are a huge help!

  • @rajan_pawar
    @rajan_pawar Жыл бұрын

    Bro literally find it useful, definitely going to follow this roadmap! Thanks a lot :)

  • @carlo6195
    @carlo6195 Жыл бұрын

    Thank you, Darshil! Concise and very clear.

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Glad it was helpful!

  • @shaguar6439
    @shaguar6439 Жыл бұрын

    Graduated for DE, tried doing my own thing after Uni, sales.. Gearing back to break into DE. Need a big refresher, it’s been almost 2 years now. Thank you so much 🙏🏽 Good luck all of u Gs

  • @gavinkalaher7314
    @gavinkalaher7314 Жыл бұрын

    Great video Darshil, very consise and informative. I've downloaded the roadmap, many thanks. Subbed!

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Awesome, thank you!

  • @jisookim9719
    @jisookim9719 Жыл бұрын

    Literally so clear, im a bioinformatics student who’s trying to transition into data engineering. This makes it less intimidating and now i know where to start

  • @vinisharma3459
    @vinisharma3459 Жыл бұрын

    Such a great explanation, thank you so much. 😊

  • @mohammedlashin9517
    @mohammedlashin9517 Жыл бұрын

    Thanks a lot Darshil for this amazing and detailed video 🙏 You sorted all the details in a very easy way which encourages anyone to start over. Thanks again brother ❤️

  • @sagarmoy_unscripted
    @sagarmoy_unscripted Жыл бұрын

    Awesome roadmap and thanks for making great content... really appreciate it 🙏

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    My pleasure!

  • @ashishkumar-nx6nv
    @ashishkumar-nx6nv Жыл бұрын

    Extremely helpful content. For last one year I was trying to create a roadmap but wasn't able to get a clear picture. Appreciate the work very nicely present without extra jargons. ❤❤

  • @blockafter
    @blockafter Жыл бұрын

    Thank you for the great content, Darshil!

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    My pleasure!

  • @romanchirkov5441
    @romanchirkov5441 Жыл бұрын

    Wow, being a Product Analyst in the industry since 2020, the level of {value / time} metric here is just phenomenal. Thank you!

  • @sumedhpatil
    @sumedhpatil Жыл бұрын

    Thanks a lot Darshil,what an explanation

  • @InglesConConfianza
    @InglesConConfianza8 ай бұрын

    GREAT resource! Thank you.

  • @vishruth1997
    @vishruth1997 Жыл бұрын

    Thanks a lot Darshil , as for someone who might be new or even for someone who have intermidiate skills in data science stack , this video is such a boon and makes less intimidating in the learning process. Hands down one of the best data engineering roadmaps i have seen .

  • @mahteenbash
    @mahteenbash Жыл бұрын

    This was actually very helpful. Cleared so many things in my mind about the different technologies. thank you.

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Glad it was helpful!

  • @SurajYadav-bc5id
    @SurajYadav-bc5id11 ай бұрын

    Thanks a ton brother for simplifying the path as much for a beginner. Now this data engineering feels less overwhelming to me. And I feel I can easily start, have clear and concise picture. Everyone you see on internet lists each and every tool and technology as if a 4 year engineering course, like telling you to learn almost everything as a beginner. Even asking to learn legacy tools which hardly used in modern day, these are just burden on a beginner. We can learn fewer technologies build few projects and can call ourselves a data engineer and then we can build upon that. Thanks again❤

  • @msnrao123

    @msnrao123

    11 ай бұрын

    Second with you

  • @abundantlife4973
    @abundantlife4973 Жыл бұрын

    Thank u so much darshil... I'm speachless 😇

  • @revanthsrirangaraju8863
    @revanthsrirangaraju8863 Жыл бұрын

    Thank you, you're video was very helpful 💯

  • @shivsuthar2291
    @shivsuthar2291 Жыл бұрын

    Thank Darshil. This is very helpful

  • @silvertechnolo3958
    @silvertechnolo3958 Жыл бұрын

    This is the most helpful video, all the other videos make it sound like I have to learn every single framework and do so much. I am A computer Engineer graduate, I think I will start at Data warehouse and pi spark since I am very comfortable with Python and Java (and C but that's just hardware stuff)

  • @luisornelas450
    @luisornelas4506 ай бұрын

    Thank you! Exactly what I need to have an idea where to start.

  • @gourabguha3167
    @gourabguha3167 Жыл бұрын

    Thank you for the awesome video!!

  • @hameedma2829
    @hameedma2829 Жыл бұрын

    It was great and worthwhile to watch the amazing video on data engineering and right path suggested

  • @saiabhinay1064
    @saiabhinay1064 Жыл бұрын

    Awsome explination. Keep Going!

  • @muthupriyadharshinim9033
    @muthupriyadharshinim903310 ай бұрын

    Thank you so much for making this. Very helpful.

  • @user-en1eh5ux7d
    @user-en1eh5ux7d11 ай бұрын

    This video is so helpful. Thank you.

  • @pushkarratnaparkhi2205
    @pushkarratnaparkhi2205 Жыл бұрын

    Thanks a lot for making it super easy for us.

  • @eduardofreitas2158
    @eduardofreitas2158 Жыл бұрын

    I am a person who avoids roadmaps and "cookbooks", because I believe there's a different path for everyone. So I consult data engineers colleagues and read articles to get recommendations and I take only what interests me. And I got to be honest with you, that's the best and most real roadmap I've seen. You spoke about all tools I've seen in the market and books I've been reading, so thank you. I'm really glad to be here. That's a great channel!

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Thanks man means a lot

  • @thenewuser3100
    @thenewuser3100 Жыл бұрын

    Thanks for the wonderful job u doing ..

  • @vivekhadoop3359
    @vivekhadoop3359 Жыл бұрын

    Really appreciate bro. thanks for guiding professional like us since its almost 10 year i m into software development lifecycle and there is no as such carrier as well salary growth. so wanted to shift my carrier in data engineering and your videos will really help me to move in that direction. I hope u also provide practical project video which will bost our handson skill while working in real time projects

  • @princeagbebo4374
    @princeagbebo4374 Жыл бұрын

    Thanks! This is really helpful

  • @user-yv1le4jg3d
    @user-yv1le4jg3d Жыл бұрын

    Great concise and informative

  • @beloyalroyal
    @beloyalroyal Жыл бұрын

    Thank you so much for perfect roadmap God bless

  • @samikshachoudhary5597
    @samikshachoudhary5597 Жыл бұрын

    Great video very helpfull, Thank You so much for your efforts 🙏🧿

  • @machhindrasmathematics2551
    @machhindrasmathematics2551 Жыл бұрын

    Thank you Darshil for the Direction...

  • @k.h.p.9862
    @k.h.p.9862 Жыл бұрын

    Thank you for this insightful video.

  • @bhumikalalchandani321
    @bhumikalalchandani321 Жыл бұрын

    Thanks darshil for useful insights

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Welcome!

  • @WadieGamer
    @WadieGamer5 ай бұрын

    Well explained, thanks for this.

  • @vinodgujja4268
    @vinodgujja4268 Жыл бұрын

    Thanks!

  • @vishnuvardhan9082
    @vishnuvardhan908211 ай бұрын

    Lovd it. Much helpful.

  • @vamsikrishnaaliveli938
    @vamsikrishnaaliveli938 Жыл бұрын

    Thank you so much by clearing the way to data engineering

  • @ravitejachilakamarthi1017
    @ravitejachilakamarthi101711 ай бұрын

    Amazing video on where, what and how to start if you are looking to pursue a career in data engineering. Thank you.

  • @pedja2518
    @pedja2518 Жыл бұрын

    Great content!

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Thank you

  • @Kshashank427
    @Kshashank4272 ай бұрын

    Dil se shukriya, very clear now

  • @rayanskingdom6029
    @rayanskingdom6029 Жыл бұрын

    Great delivery of information 👌👍 Only thing is looking at amount of information you share, please take small pauses so that the listener is able to grasp it 😊

  • @stephencallahan3390
    @stephencallahan33908 ай бұрын

    Thank you!

  • @MrBhardwaj03
    @MrBhardwaj03 Жыл бұрын

    Where I can find the doc file you showed at the end of the video?

  • @thedailyepochs338
    @thedailyepochs338 Жыл бұрын

    Awesome work

  • @vedantwankar7332
    @vedantwankar73329 ай бұрын

    You made it so easy to understand.....kudos to you brother😇😇

  • @user-gg7ud9zt9f
    @user-gg7ud9zt9f Жыл бұрын

    Hi Darshil, I am an analyst with 8+ years experience who started career after a data science certification. Due to lack of programming skills I tried to escape core tech jobs and kept dabbling with tools based jobs and learnt several tools but was never consistent. After much self retrospection and with the help of a mentor like friend I have decided to dig deeper into Data engineering concepts so that I can be a better analyst. I searched about some DE concepts and happened to get your uber project video in the suggestions, must agree that have become a fan of you from the very first minute (bande me kuch baat hai), here I am on your 4th video and I know I hit one of the gold mine in this area. I would love to catch up with you, share and learn more in this Data world. Finally, yes I agree your approach of learning and your choice of tools and platforms are excellent, I suggest the same to everyone... hopefully this comment finds you and we can catch up soon.

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Welcome to channel, as you go deep, you'll find more goldmine that will help you to learn many things

  • @khushbooagarwal6172
    @khushbooagarwal6172 Жыл бұрын

    Please share the link of databrics training.. the training sessions on databricks academy has lab sessions and I found that they are not in sync with course material. Can you please help me with this?

  • @hellbent2001
    @hellbent2001 Жыл бұрын

    Where can I get that roadmap notebook you shared at the end of video.

  • @mohamedfyrose409
    @mohamedfyrose409 Жыл бұрын

    Your contents never disappoints. Thanks for an another awesome material

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Thank you for the support

  • @SharmanjoshiBlog
    @SharmanjoshiBlog Жыл бұрын

    Very Nice being and doing fabulous work .

  • @animalbubble4487
    @animalbubble44879 ай бұрын

    Excellent video , you just broke it down for me and i will start my data Engineering journey now ..Thank you ... Honestly this is the best and simplist videoo you will see on youtube

  • @DarshilParmar

    @DarshilParmar

    9 ай бұрын

    Glad it was helpful!

  • @ankandatta4352
    @ankandatta4352 Жыл бұрын

    Currently working with dbt. What are the options going forward??

  • @abhishekkadam3891
    @abhishekkadam3891 Жыл бұрын

    Hi darshil , how much level of coding needed for a daga engineering

  • @anonymousgal100
    @anonymousgal100 Жыл бұрын

    @@DarshilParmar At what point of learning journey, I should start applying for job/internship if would like to switch career from web developer to data engineer?

  • @jaekim1717
    @jaekim171711 ай бұрын

    Thank you very much. was simple explanation but this is best

  • @jerome8471
    @jerome8471 Жыл бұрын

    Is the CS50P preferred since it focuses on python?

  • @nyakakun3398
    @nyakakun339810 күн бұрын

    Спасибо, бро!

  • @girijavarma5271
    @girijavarma5271 Жыл бұрын

    thank you so much 😇

  • @muktakjain8268
    @muktakjain8268 Жыл бұрын

    THANKYOU SOO MUCH SIR

  • @Jsmbpj
    @Jsmbpj Жыл бұрын

    I am fresher and what would you suggest- Azure or Snowflake with AWS?

  • @mosman260
    @mosman260 Жыл бұрын

    Do you recomend learning all 3 or just 1 progam lanugange?

  • @user-uq8dc6vs6z
    @user-uq8dc6vs6z4 ай бұрын

    Thank you so much

  • @NitinSai2909
    @NitinSai2909 Жыл бұрын

    very helpful video

  • @leopoldomolina1664
    @leopoldomolina1664 Жыл бұрын

    I think that I just recently discovered that data engineering is my roadmap to follow. But I find myself confused because all the names and terminology are very different or new from what I’ve been used to. The only common name that I have heard of is Python; the rest is completely new. Now I am doubting about Data Engineering. Any advice?

  • @syed_hussaini
    @syed_hussaini Жыл бұрын

    The shared link for the Data Engineering course is not working at all

  • @avinash7003
    @avinash7003 Жыл бұрын

    What are the softwares required for data engineer

  • @haseebkhalid4911
    @haseebkhalid4911 Жыл бұрын

    how can I get this word sheet you showed in the last section?

  • @AkshayKumar-GG
    @AkshayKumar-GG Жыл бұрын

    is there any upper hand for data analyst while transitioning and giving interviews for DE role?

  • @altunbikubra
    @altunbikubra11 ай бұрын

    great video! overall, not too specific and general depiction of the pathway. Thank you very much!

  • @makampooja6767
    @makampooja6767 Жыл бұрын

    You are simply superb and life saver

  • @rohankabadi389
    @rohankabadi389 Жыл бұрын

    Darshil i have a question for you, I am a chef and i have to prior tech knowledge i am thinking of making a career change into data engineering. Would it be possible for me to get a job later without a CS degree?

  • @sravsclassic4369
    @sravsclassic43697 ай бұрын

    best road map, very clearly explained

  • @shuvranshuhalder6623
    @shuvranshuhalder6623 Жыл бұрын

    Can I learn all those things in my windows I3 7th gen laptop?

  • @ishikadabare5567
    @ishikadabare556711 ай бұрын

    Thank-you, can you frame a course covering basics and intermediates?

  • @vishnuinsane
    @vishnuinsane Жыл бұрын

    Thanks Dharsil.. it’s awesome Map to Hit the CV’s

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    All the best

  • @kunalbedre5727
    @kunalbedre5727 Жыл бұрын

    Clear 😊

  • @shashankreddysai
    @shashankreddysai Жыл бұрын

    Top notch content

  • @goodman4093
    @goodman4093 Жыл бұрын

    Where can I get the best data engineering course online?

  • @Ganeshay-996
    @Ganeshay-996 Жыл бұрын

    Finally I got it best roadmap

  • @zaskens8083
    @zaskens8083 Жыл бұрын

    Can someone help me? I have doubts about which specialty to take in the data sector (engineer / scientist). I would appreciate any advice :)

  • @mzug9
    @mzug93 ай бұрын

    Thank you

  • @nikhilchintalapudi4932
    @nikhilchintalapudi4932 Жыл бұрын

    Idk if it's just me, but my current work as a data engineer requires me to learn a lot of dev ops principles (cloudformation/templates/helm) and networking principles (VPC/subnets/security groups). I learned python and a couple other tools like dbt, airflow, etc and came from a heavy SQL background (snowflake, sql server, ssis). Having to learn this other stuff while also getting better at python, spark, and the main parts of data engineering came as a big surprise to me.

  • @HeavensMeat

    @HeavensMeat

    Жыл бұрын

    Any good sources to learn those?

  • @nikhilchintalapudi4932

    @nikhilchintalapudi4932

    Жыл бұрын

    @@HeavensMeat nothing too great tbh, I used acloudguru which helped conceptually with some of the basic stuff but I found no good substitute for learning on the job/practicing with small projects and deployments in a personal account.

  • @JungleEd17

    @JungleEd17

    8 ай бұрын

    I've been doing infrastructure for DE for a year. In addition they have a devops team for pipeline deployment from github. So most just need to know which branch to push to. Is it a small team?

  • @AshishKumar-th8by
    @AshishKumar-th8by Жыл бұрын

    In the sheet that you shared, you mentioned about your project-01, should I do that before I get started with Big data tutorials as it is mentioned early there?

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Yes

  • @pitamdas8416
    @pitamdas841610 ай бұрын

    Will you suggest some structured course?

  • @Slimshady68356
    @Slimshady6835610 ай бұрын

    Are data scientist and data engineer same , as of my internship I did in Data scientist ,I think the validation , preprocessing step are also included , data base handling are there any other differences ?

  • @DonnyLp03
    @DonnyLp03 Жыл бұрын

    I agree that building a good foundation of fundamentals is important.

  • @SandyBoyy9214
    @SandyBoyy921411 ай бұрын

    As someone with over 12 years experience working with databases and data warehouses, i can vouch for Darshil and the quality of content he shares. I recommend any aspiring data engineer to follow him closely. This man is a master and a fantastic tutor.

  • @DarshilParmar

    @DarshilParmar

    11 ай бұрын

    I appreciate the kind words sir

  • @SandyBoyy9214

    @SandyBoyy9214

    11 ай бұрын

    @@DarshilParmar You deserve every bit of it Darshil. Keep inspiring the young engineers and contribute to the growth of this country. More power to you 🙂

  • @imranahmed-ri7iz
    @imranahmed-ri7iz Жыл бұрын

    It’s a great explanation Darshil.. now I have a clear path to go for preparations. One doubt, what about Hadoop ecosystem? Is it not necessary to learn??

  • @DarshilParmar

    @DarshilParmar

    Жыл бұрын

    Most of the companies are focusing on spark so not really, if you want then you can explore the basics and move on

  • @focus1007
    @focus10077 ай бұрын

    Subscribed 😍just found your channel and its helpful

  • @yashkothve4340
    @yashkothve4340 Жыл бұрын

    Could you please give me sheet which is you have to show last block?

  • @keneugene7714
    @keneugene7714 Жыл бұрын

    this is great

Келесі