LIKE LIKE LIKE LIKE!!!!! Interested in Learning Apache Spark in-depth with Databricks, I have created a detailed course here: datavidhya.com/courses/apache You can directly enroll in the best combo package Python, + SQL + Data Warehouse Snowflake + Apache Spark with Databricks Get it here: Combo Package: com.rpy.club/pdp/yYnEMzLOX?plan=6607b619c69cf00b7b934479… USE CODE: COMBO50 for 50% off

@abhishekbatham5588

8 күн бұрын

Can you create a weither analising with python please provide that want to learn

I am so relieved that there is someone who depicts a “complete” pipeline for projects that are not just real-world but also easy to comprehend, without loosing their innate complexity. Thanks alot for your contribution

Such an amazing project to learn Apache Spark with Databricks! I learned so much, and the clarity of concepts was incredible. Thank you so much, Darshil! Totally going for your Combo Course!! 🙌

Amazing... This architecture is applied in more real-time projects

If youve done Sql very well then all you need here is just learning the few syntax differences in spark and Sql.

Hi @DarshilParmar thank you for all these videos. It's too good!!!!. I am a beginner, I really love it. I just started yesterday. You never let me blink my eye.

A very good project in a small project alot of learning ,this is called project based learning ❤🎉

Wow, this video is incredibly informative! I really appreciate how clearly it explains complex concepts. The visuals are engaging and make it easy to follow along. I'm excited to dive deeper into Spark after watching this. Keep up the great work!

Thanks a lot man. Much needed video.

I just love all your videos. Take love from Bangladesh❤

Thanks Darshil this was very imformative and a Good learning Project journey for me as Data Engineer! Kudos please keep posting such Projects!

Great explaination!! Thank you.

thankyou Darshil

Wonderful insights into the Spark, never got distracted and fully engaging.

@DarshilParmar

3 ай бұрын

Thank you

its a great project!

Amazing work Darshil bhai Loved the project

@DarshilParmar

3 ай бұрын

Thank you so much 😀

Great initiative . Thank you so much. Please take care of the audio. It's too low !!

Loved the Project Darshil Bhaiya I'm a Beginner and I'm loving it

@DarshilParmar

3 ай бұрын

Let's go

Thank you bro❤

Wonderful video

This was a nice project. Thanks!

@DarshilParmar

3 ай бұрын

Glad you liked it!

Amazing!

Amazing content

Right video at right time. Thanks @darshil bai🤩

@DarshilParmar

3 ай бұрын

You are welcome

Amazing content as usual.

@DarshilParmar

3 ай бұрын

Much appreciated!

Now this is what data enthusiasts need most people build the project directly on power bi or SQL without giving complete understanding.

@DarshilParmar

3 ай бұрын

Thank you

56:57 correction when (col("batting_hand").contains("Left"), "Left-Handed").otherwise ("Right-Handed")

Hello, really nice videos. I really like how you teach, and I am interested in starting the spark databricks course. I have knowledge of SQL and Python but no previous knowledge of Snowflake. Can I still do the spark and databricks course without snowflake??

Is Amazon s3 used for data modelling?

Hi Darshil, Could you please share your Data Vidhya Notes as a pdf. While enrolling it's asking more amount. Please help me on this. Excellent video.

We can do the same thing in sql as well. Why to use spark?

Plz create video on pyspark unittesting and debugging

one thing in your explanation i observed is you are crisp and right to the point interms of explanation sir....if you ask me to explain analytically ...more value delivered in the least amount of time without any deviation.....great work sir... will learn more from you

@DarshilParmar

3 ай бұрын

Thank you very much :)

Hi Sir How are you? Sir, it is possible to fetch datasets from Kaggle using Azure Data Factory.? with azure function it is possible. Here's how?

Bro, could you please provide us this obsidian whole notes link for this project……..

How can we round off pin pyspark(liek if I want to round off a value to two decimal places) how is that possible?

Nice work brother

@DarshilParmar

3 ай бұрын

Thank you! Cheers!

Thank you brother ❤❤ love from Nepal 💗💗

@DarshilParmar

3 ай бұрын

Always welcome

how to get a data enginner internship and how much do i get to know for internship ?

Bhaiya your courses are too expensive I also want to learn can you take down the price of the combo package course......pls....!!!!

Hi Darshil, thanks for the insightful videos, is it okay to use Macbook Air for data engineering?

@DarshilParmar

3 ай бұрын

Yes

Nice video!!, what is the software you're using in your iPad for this presentation?

@DarshilParmar

3 ай бұрын

Good notes

provide the amazon s3 bucket link

Hi Darshil, Can i get a notes for python if i buy course, please answer

How much python is needed ? I am just starting 🙏

How can i create an aws account without a credit card please reply

Thanks for sharing such a lovely course on EDA using Apache Spark. Please could you correct the code at 56:13 where the "batting_hand" contains "left" rather it should be "Left" as the batting_hand column contains like "Left-xxxxx".

@adilmajeed8439

2 ай бұрын

@DarshilPamar Thanks again for sharing the project along with the solution. I was able to convert the same project to Microsoft Fabric. Lots of learning ...

Date columns are appearing as null. BoolType columns are also appearing as null. Can you resolve that?

I loved your content thanks for sharing and I confused to choose which database is good MySQL or PostgreSQL to learn. Can Anyone suggest me

@aviatorifeanyi4239

3 ай бұрын

I will recommend PostgreSQL, MySQL is also cool. Little difference in syntax between the two

how do i create a account if i am still a student

how can i use your bucket??

Bhai aap kha rhte ho milna h aapse

can you provide me your s3 bucket url of ipl analysis so i can use in my project, because i donot have aws account

Bro in Olympic data analysis config code in data bricks gave me error saying null value exception

@DarshilParmar

3 ай бұрын

Issue might be with keys, lot of people copy secretID but you need to copy SecretValue

@ Darshil Parmar Thank you. As a fresher, Can I try the jobs in the data engineering field in USA?

@DarshilParmar

3 ай бұрын

It is possible

can i make this project using jupyter notebook as well or there any particular reason for using Databricks (just asking) ?

@DarshilParmar

2 ай бұрын

You can, you need to configure spark with jupyter notebook

👍

❤

Once you’ve built a portfolio project, how do you store and present it?

@DarshilParmar

2 ай бұрын

Github

Hii.. Is there any way to contact you?

how to copy address of the ball_by_ball table from dataset ?

@DarshilParmar

3 ай бұрын

Just use s3 path

Hey @DarshilParmar, I didnt get why you consider only 'run_scored' column while calculate #Aggregation :Calculate the total and avg runs scored in each match and inning. In our dataframe, 'ball_by_ball_df', we record details like this: 1. When a bowler bowls a no-ball and the batsman scores 4 runs on that ball, it results in a 'run_scored' entry of (4) and an 'extra_runs' entry of (1) in the respective columns. 2. If a bowler bows wide, it's marked as (0) in the 'run_scored' column and (1) in the 'extra_runs' column. Now, when calculating the total runs for a match and innings, we need to add up both the 'run_scored' and 'extra_runs' columns to get the accurate total."

@DarshilParmar

2 ай бұрын

I kept saying in the video, goal of the video is not to get business logic right but to teach how to use tech

@BigDataEngineerSchool

2 ай бұрын

@@DarshilParmar Ya I forget..thanks to rply❤❤

Great Thank you

@DarshilParmar

3 ай бұрын

You are welcome

What do you use to build the Architecture Diagrams?

@DarshilParmar

22 күн бұрын

Google Slides

@techxball

22 күн бұрын

@@DarshilParmar Thank you

this looksk like a basic projects i dont think this is enought ot put it in resume!

@DarshilParmar

3 ай бұрын

You can never put KZread projects on resume, 100k+ people do these project do you think you can stand out by doing these project? These projects are for learning and upskilling, only project you put in your resume is something that you create by yourself

Spark isnt distributed rather than parallel?

@DarshilParmar

3 ай бұрын

It is both

Can we replicate this project in GCP entirely? Please advice Darshil.

@DarshilParmar

3 ай бұрын

Yes use GCS, DataProc, BigQuery

In your course are you going to teach pyspark or spark sql? Which one?

@DarshilParmar

14 сағат бұрын

Both

Why Having Count is > 120

Do we have its source code available?

@DarshilParmar

3 ай бұрын

Check description

bring Airflow course

@DarshilParmar

3 ай бұрын

Next on the pipeline

Math bhi kuch Ani chiye ka ? 🤔

@DarshilParmar

3 ай бұрын

Basic college level

Hey Im getting error while reading form s3

@DarshilParmar

3 ай бұрын

What's the error?

@adityatomar9820

3 ай бұрын

@@DarshilParmar hey , I solved it ! It was access denied error...made my bucket public and it works now🤗

Can you please share your notes?

@DarshilParmar

3 ай бұрын

Notes are part of my courses, internal document, used in video to explain basic stuff

@sateeshkumar2698

3 ай бұрын

@@DarshilParmar Oh ok, If possible can you sell notes alone please

@phaddu7737

3 ай бұрын

@@DarshilParmar Hey, interested in the standalone Python course Darshil. Discounts coming any time soon.

@sateeshkumar2698

2 ай бұрын

Mr also bro, are u purchased?​@@phaddu7737

First comment 🎉❤

@DarshilParmar

3 ай бұрын

Let's go!

DARSHIL = 7 letters #Thalaforareason

@DarshilParmar

3 ай бұрын

haha