Big Data Analysis with Scala and Spark

Big Data Analysis with Scala and Spark

Cluster Topology Matters!

Cluster Topology Matters!

Reduction Operations

Reduction Operations

Pair RDDs

Pair RDDs

Latency

Latency

Introduction & Logistics

Introduction & Logistics

Joins

Joins

Partitioning

Partitioning

Optimizing with Partitioners

Optimizing with Partitioners

Wide vs Narrow Dependencies

Wide vs Narrow Dependencies

Spark SQL

Spark SQL

DataFrames (1)

DataFrames (1)

DataFrames (2)

DataFrames (2)

Datasets

Datasets

Пікірлер

  • @user-xl9fs3tp8y
    @user-xl9fs3tp8y11 ай бұрын

    Really helpful !!!

  • @bres6486
    @bres6486 Жыл бұрын

    At most one key value pair per id per node (not key value pair per node as far as I understand) after using the reduceByKey().

  • @vikastangudu712
    @vikastangudu712 Жыл бұрын

    you are awesome.

  • @mateusznowakowski6805
    @mateusznowakowski6805 Жыл бұрын

    Great video

  • @bigdataenthusiast
    @bigdataenthusiast Жыл бұрын

    simply great

  • @ddoshi39
    @ddoshi39 Жыл бұрын

    Thank you so much

  • @damianoderin4874
    @damianoderin48742 жыл бұрын

    Awesome course. Thanks a lot!

  • @rydmerlin
    @rydmerlin2 жыл бұрын

    How can I combine queries to multiple data sources and get one result?

  • @rydmerlin
    @rydmerlin2 жыл бұрын

    Why does it flicker so much?

  • @balanceresume2802
    @balanceresume28022 жыл бұрын

    🤩😍🥰 heather miller

  • @Manapoker1
    @Manapoker12 жыл бұрын

    thx you for this video, it helps a lot! <3

  • @WaterWheel360
    @WaterWheel3602 жыл бұрын

    commenting for the KZread algorithm

  • @ashwinichandran8839
    @ashwinichandran88393 жыл бұрын

    Wonderful explanation.... waiting for many videos from you on different technologies like HIVE and PySpark

  • @ManikantGoutamReal
    @ManikantGoutamReal3 жыл бұрын

    this is god-level video. thanks a lot.

  • @user-ep2vw2ss5y
    @user-ep2vw2ss5y3 жыл бұрын

    the only sorry that i cant get english

  • @souravbanerjee5744
    @souravbanerjee57443 жыл бұрын

    can you share the link of the scala course referred often in this series ?

  • @nageshbs8945
    @nageshbs89453 жыл бұрын

    we can't say database are structured, many no sql database do not support schema

  • @Mryajivramuk
    @Mryajivramuk3 жыл бұрын

    Very impressive mentor you are....pls do full series on spark and scala ...and be a part of our journey.

  • @madhu1987ful
    @madhu1987ful3 жыл бұрын

    Coalesce is a wide transformation? Can u pls explain in detail. Thanks

  • @andys7596
    @andys75963 жыл бұрын

    So many videos in other channel but this one after so many years still has best value content. Thank you !

  • @LivenLove
    @LivenLove3 жыл бұрын

    What are the deciding factors for number of partitions

  • @LivenLove
    @LivenLove3 жыл бұрын

    Only channel where a don't increase playback speed

  • @avsbharadwaj8190
    @avsbharadwaj81903 жыл бұрын

    why there is no mapper side optimisation for the groupByKey operation?

  • @underlecht
    @underlecht3 жыл бұрын

    Hello, 1:30 for "fastest" calculation you apply shuffling in line 3, and after that you measure the duration. Why don't you include shuffling to duration? Data preparation also takes time. Unless you mean "shuffle once and for all", but in reality it is hard to imagine that you will be grouping by one column only in your calculations. Thanks.

  • @narendernegi7493
    @narendernegi74933 жыл бұрын

    Amazing.

  • @gothamsudheer4751
    @gothamsudheer47513 жыл бұрын

    Your teaching skills excellent. You know how to teach.Thank you so much......

  • @oguzhan2393
    @oguzhan23933 жыл бұрын

    finally, I found good videos about spark and scala and she is using crystal clear english

  • @yangmingwang160
    @yangmingwang1603 жыл бұрын

    You make the best video among the Spark tutorials on KZread, thank you!

  • @aspait
    @aspait4 жыл бұрын

    we can use pre-partition in map RDD(like hash and range) how can I use it in Dataframe?

  • @aneksingh4496
    @aneksingh44964 жыл бұрын

    Please keep posting new videos on spark and scala ... Your videos are awesome 👍

  • @DatNguyen-ry1vr
    @DatNguyen-ry1vr4 жыл бұрын

    Gold!!

  • @aneksingh4496
    @aneksingh44964 жыл бұрын

    absolutely great .... please add some more videos on spark real time use cases ... thanks

  • @pratikkawalgikar4839
    @pratikkawalgikar48394 жыл бұрын

    The concept is now clear for me after searching all over the net from last 3 months. Thanks a lot. Your videos are very simple to understand. Please upload more on spark as I have finished watching all your videos and they are simply superb.

  • @skms31
    @skms314 жыл бұрын

    ❤️ From India

  • @WisdomWaves33492
    @WisdomWaves334924 жыл бұрын

    How Millions of data analysed by Spark?....

  • @mrkrish501
    @mrkrish5014 жыл бұрын

    Excellent

  • @gauravlotekar660
    @gauravlotekar6604 жыл бұрын

    would you be able to share the PPT ?

  • @lishi6858
    @lishi68584 жыл бұрын

    The best formation of spark !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

  • @jacobkim9856
    @jacobkim98564 жыл бұрын

    Best

  • @jayfix192
    @jayfix1924 жыл бұрын

    thank you

  • @careymain3036
    @careymain30364 жыл бұрын

    how to you filter by date like between 1-1-2020 and 1-30-2020 from a parquet file the date field is a string

  • @careymain3036
    @careymain30364 жыл бұрын

    how to you filter by date like between 1-1-2020 and 1-30-2020 from a parquet file the date field is a string

  • @_sr
    @_sr4 жыл бұрын

    The best explanation I have ever seen.

  • @kiraninam
    @kiraninam4 жыл бұрын

    the teacher has remarkable concepts, Hi teacher how can i join your course if you are offering. I am looking for Spark training

  • @kiraninam
    @kiraninam4 жыл бұрын

    very impressive concept based knowledge. greate job.

  • @sunitareddy8717
    @sunitareddy87174 жыл бұрын

    Your explanation is amazing which I couldn't get even spending hrs.

  • @JohnDoe-zc4mu
    @JohnDoe-zc4mu4 жыл бұрын

    Holy cow, u explained in 12min something I had to understand in 1 hour from other videos.

  • @rizvihasan6459
    @rizvihasan64594 жыл бұрын

    This channel is one of the best tutorial i have seen in youtube. Big thanks and I really appreciate it.

  • @hiteshbitscs
    @hiteshbitscs4 жыл бұрын

    why all mediocre content in HD and one of the most imp in 360p... sick

  • @sivakannan23
    @sivakannan235 жыл бұрын

    Excellent one. Thanks.