What no one tells you about Data Engineering Real time Project? Reality and Facts in 15 minutes
Ғылым және технология
#DataEngineering #ProjectCreation #RealTimeDataEngineering #DataEngineeringTutorial #DataEngineeringProject #PreImplementationPhase #ImplementationPhase #PostImplementationPhase #DataEngineers #SolutionArchitects #DevOpsEngineers #InfrastructureEngineers #ProjectManagers #ModernDataPlatform #Terraform #CICDPipelines #DataPlatformMaintenance #DataEngineeringCareer #DataEngineeringInterviewTips #AzureDataFactory #Databricks #AWSDataEngineering #GCPDataEngineering #DataPlatformArchitecture #InfrastructureAsCode
Welcome to my channel! In today's video, we’re diving deep into the creation of a real-time Data Engineering project from scratch. This comprehensive guide covers everything you need to know, from the initial discussions to the final implementation and maintenance. Unlike many tutorials you might have seen, we’ll explore every detail of the process, making it a must-watch for anyone looking to break into the Data Engineering field.
What We’ll Cover:
1. Introduction
2. Key Roles Involved
3. Pre-Implementation Phase
4. Implementation Phase
5. Post-Implementation Phase
Throughout the video, we’ll provide detailed insights into each phase, emphasizing the critical role of Data Engineers across all stages. By the end, you’ll have a thorough understanding of the complete lifecycle of a Data Engineering project, equipping you with the knowledge to excel in your career or ace your next interview.
If you find this video helpful, please like, share, and subscribe to my channel for more insightful content. Let's get started on this exciting journey into the world of Data Engineering!
Cheers, and see you in the video!
TimeStamp:
0:00:00 - Intro
0:00:59 - Key Roles Involved
0:02:25 - Pre-Implementation Phase
0:07:17 - Implementation Phase
0:12:27 - Post-Implementation Phase
- - - Book a Private One on One Meeting with me (1 Hour) - - -
www.buymeacoffee.com/mrktalks...
- - - Express your encouragement by brewing up a cup of support for me - - -
www.buymeacoffee.com/mrktalks...
- - - Other useful playlist: - - -
1. Microsoft Fabric Playlist: • Microsoft Fabric Tutor...
2. Azure General Topics Playlist: • Azure Beginner Tutorials
3. Azure Data Factory Playlist: • Azure Data Factory Tut...
4. Databricks CICD Playlist: • CI/CD (Continuous Inte...
5. Azure Databricks Playlist: • Azure Databricks Tutor...
6. Azure End to End Project Playlist: • End to End Azure Data ...
7. End to End Azure Data Engineering Project: • An End to End Azure Da...
- - - Let’s Connect: - - -
Email: mrktalkstech@gmail.com
Instagram: mrk_talkstech
- - - About me: - - -
Mr. K is a passionate teacher created this channel for only one goal "TO HELP PEOPLE LEARN ABOUT THE MODERN DATA PLATFORM SOLUTIONS USING CLOUD TECHNOLOGIES"
I will be creating playlist which covers the below topics (with DEMO)
1. Azure Beginner Tutorials
2. Azure Data Factory
3. Azure Synapse Analytics
4. Azure Databricks
5. Microsoft Power BI
6. Azure Data Lake Gen2
7. Azure DevOps
8. GitHub (and several other topics)
After creating some basic foundational videos, I will be creating some of the videos with the real time scenarios / use case specific to the three common Data Fields,
1. Data Engineer
2. Data Analyst
3. Data Scientist
Can't wait to help people with my videos.
- - - Support me: - - -
Please Subscribe: / @mr.ktalkstech
Пікірлер: 42
Hi Everyone! Creating this video took a lot of time and effort, and I truly hope you find it helpful. If you did, please like, share, and subscribe to my channel. Your support means the world to me and motivates me to create more valuable content for you. Thank you! 😊 Follow my LinkedIn account for regular updates: www.linkedin.com/in/mrk-talkstech/
Hi, This presentation is very different from your old videos. This presentation points out the data engineer's role in the project implementation part. I appreciate you creating these videos for the younger communities to learn and understand the environment of data science, data engineering, and data analysis. Thank you for your service.🙏🙏🙏
You nailed it again.. just awesome.
Now words to appreciate...this presentation👏
Much appreciated work sir
Very useful video and now I get an idea of what really happens and how are project is developed from scratch! Highly appreciated to create this for us. 🙏🏻
@mr.ktalkstech
14 күн бұрын
Thank you so much :)
Thats so insightful thanks sir🎉
Thankyou I got some idea how it's working in industries
Really helpful and must needed video.please make one more video regarding how data engineer daily work activities, day to day tasks, meetings, and how to handle all these. Please make one video.
Can u pls make one big data engineering couese for aspiring professional please sir?
Great video! This really clarifies how live data engineering environments operate and the various phases involved. Could you please create more real-time complex data engineering projects that we can add to our portfolios and resumes? Your guidance would be immensely valuable!
Such a beautiful content…Thank you very much
@mr.ktalkstech
14 күн бұрын
Thank you so much :)
Thanks for explaining the process in detail.
@mr.ktalkstech
14 күн бұрын
Thank you so much :)
thank you very much for this video . its helpful
@mr.ktalkstech
15 күн бұрын
Glad to hear that :)
@mr.k does data engineers need to work on IAC as some job requirements are asking for Terrraform , docker and kubernetes as well . Is it true that for 10 DE only 2 - 3 devops/platform engineers are maintained by organisations??
please make some more insightful videos on DE, it would be really helpful for freshers and less experienced. please also make a video on Tools data engineer used for ducementation and day to day activity tracker etc etc
Can data engineering be an implementation specialist or support specialist of the project like dba does
Very Helpful. Thanks. Please create more end-to-end projects on Azure Stack. Also, Please take data which has issues so that the project flow can also show edge cases.
@mr.ktalkstech
14 күн бұрын
Thank you so much, sure will try :)
sir can you please make a SAP data engineering project and a apache data engineering tools project playlist
What is the next step for data engineering
A suggestion/request : Can we get on a zoom seminar where you start a data engineering project from scratch and build it end-to-end while answering question and explaining reasons behind a certain approach taken and the other ones avoided ?
@mr.ktalkstech
14 күн бұрын
Sure, will try :)
Found gold
@mr.ktalkstech
14 күн бұрын
Thank you so much :)
Hey I would like to know where you are from
@mr.ktalkstech
14 күн бұрын
Chennai :)
@manibaddireddy5477
14 күн бұрын
@@mr.ktalkstech great work man I loved your way of teaching You should be in a great position in future.
@mr.ktalkstech
14 күн бұрын
@@manibaddireddy5477 Thank you for your kind words :)
@manibaddireddy5477
14 күн бұрын
@@mr.ktalkstech I hope you had a great experience with various domains so why can't you share domain knowledge because technical knowledge what you are sharing is different but think about domain knowledge sharing.
Please start training for data engineers
@mr.ktalkstech
15 күн бұрын
Sure, will do :)
@B_maheshkumar_B
15 күн бұрын
Please let us know as soon as possible the course has been initiated@@mr.ktalkstech
I am in third phase and struggling to go to second phase
@mr.ktalkstech
14 күн бұрын
You will soon, try to look for some startup :)
@chirunjeevichiru8308
14 күн бұрын
@@mr.ktalkstech thank you for your response, I have gone through your end to end project and did practice I got good understand can you make video series focusing on complex pipelines with parametrization and difference between full load and incremental load and creating complex whare house modeling in your up coming real time project which will helpful to understand complexity.
@mr.ktalkstech
14 күн бұрын
@@chirunjeevichiru8308 Sure, will do in the future :)
Can u share ur LinkedIn profile