Google File System - Paper that inspired Hadoop

Distributed File Storage made by Google around 2003
Channel
----------------------------------
Complex concepts explained in short & simple manner. Topics include Java Concurrency, Spring Boot, Microservices, Distributed Systems etc. Feel free to ask any doubts in the comments. Also happy to take requests for new videos.
Subscribe or explore the channel - / defogtech
New video added every weekend.
Popular Videos
----------------------------------
What is an API Gateway - • What is an API Gateway?
Executor Service - • Java ExecutorService -...
Introduction to CompletableFuture - • Introduction to Comple...
Java Memory Model in 10 minutes - • Java Memory Model in 1...
Volatile vs Atomic - • Using volatile vs Atom...
What is Spring Webflux - • What is Spring Webflux...
Java Concurrency Interview question - • Java Concurrency Inter...

Пікірлер: 84

  • @adiglasman2198
    @adiglasman21984 жыл бұрын

    You have managed to take the written article published by google and explains Google File System and MapReduce in a very good, clear way so anyone can consume it. good job!

  • @siddheshswnt
    @siddheshswnt4 жыл бұрын

    This is how one should teach. Loved it!

  • @nitinaggarwal8278
    @nitinaggarwal82783 жыл бұрын

    Best explanation of GFS on KZread. This is how one should teach. Wish the channel had posted more such videos.

  • @mavbloopy
    @mavbloopy5 жыл бұрын

    Thank you for getting into distributed systems realm. Would love to see some videos about the workings of cassandra.

  • @srijanya2112
    @srijanya21124 жыл бұрын

    This explanation and the voice clarity is too good, thanks a lot for the video

  • @SayanInMoves
    @SayanInMoves4 жыл бұрын

    amazing video. the finest tutorial of GFS on youtube.

  • @mfaraday4044
    @mfaraday40444 жыл бұрын

    Now I realsed that my teacher must have watched this video before lecture .

  • @atharvajadhav8598
    @atharvajadhav85984 жыл бұрын

    Excellent video clearing the basics of GFS.

  • @gauravsah7780
    @gauravsah77804 жыл бұрын

    Your videos are crisp and very clear. Thanks. Keep up the good work😊👍

  • @DanielSmith-uj7rr
    @DanielSmith-uj7rr3 жыл бұрын

    Very well systematically explanation.! Great Work! Thank you. God bless you.

  • @chandandastanti7557
    @chandandastanti75573 жыл бұрын

    Your explanation is GREAT.

  • @salma-amlas
    @salma-amlas8 ай бұрын

    high quality video. your explanations are clear as crystal

  • @aakashkhopade4666
    @aakashkhopade46663 жыл бұрын

    Perfect explanation .Thank you for this.

  • @nikhilmakam8794
    @nikhilmakam87942 жыл бұрын

    Thanks for the great content. But I haven't seen any recent videos from you, it'd be great if you start making these videos again, they're just wonderful.

  • @ScribbleDribble
    @ScribbleDribble3 жыл бұрын

    Very clear explanation. Thank you!

  • @prajaktakhandve7501
    @prajaktakhandve75013 жыл бұрын

    Very well explained and articulated.

  • @piyushkumar-wg8cv
    @piyushkumar-wg8cv Жыл бұрын

    So a simple explanation. Amazing.

  • @ChetanKulkarni1994
    @ChetanKulkarni19944 жыл бұрын

    Very nicely explained.Thanks a lot :)

  • @MartinCharles
    @MartinCharles3 жыл бұрын

    Can't wait for the BigTable and DynamoDB episodes!! Also can you do one on pagerank?

  • @adityalingam2635
    @adityalingam2635 Жыл бұрын

    Very well explained. Thanks for the content and effort

  • @mukeshrathaur
    @mukeshrathaur4 жыл бұрын

    Excellent Explanation sir. Loved it. Wanted to learn more on it. I am now big fan of yours. Watched most of your videos and trying to read as much possible from you sir. Please consider to share more videos on this topic.

  • @DefogTech

    @DefogTech

    4 жыл бұрын

    Thank you so much for your kind words! Planning to do more videos soon

  • @SteveLoughran
    @SteveLoughran2 ай бұрын

    FWIW the google paper left out some of the details you needed for full resilience. We had to learn those through things going wrong -e.g the infamous facebook cascade failure where, once a critical mass of servers went offline, the recovery work took the rest down

  • @talirabetti8066
    @talirabetti80665 жыл бұрын

    Thank you! a very clear explanation.

  • @nkadoor
    @nkadoor2 жыл бұрын

    Very nicely explained. Thank you.

  • @shreyas6589
    @shreyas65894 жыл бұрын

    Great explanation! Thanks.

  • @arizavala5297
    @arizavala5297 Жыл бұрын

    Amazing video, super clear.

  • @whattheydoexactly7794
    @whattheydoexactly7794 Жыл бұрын

    Amazing video. Thank you so much

  • @SameerSrinivas
    @SameerSrinivas3 жыл бұрын

    Great video. Thanks for making. There is a typo in the last slide about chunk size being 64kb instead of 64mb :)

  • @bhimanijanki
    @bhimanijanki Жыл бұрын

    Great job!

  • @magnarwium8520
    @magnarwium85204 жыл бұрын

    Nice! Very easy to follow

  • @anirvangoswami
    @anirvangoswami6 ай бұрын

    Such a good explanation.

  • @lenso2010
    @lenso20102 жыл бұрын

    Well explained ,thank you.

  • @prabudasv
    @prabudasv3 жыл бұрын

    Very well explained , thanks

  • @eko100200
    @eko100200 Жыл бұрын

    Great explanation.

  • @jorgedfbranco
    @jorgedfbranco3 жыл бұрын

    Very good explanation!

  • @danyald4880
    @danyald48803 жыл бұрын

    Exceptional sir.

  • @nishantparmar
    @nishantparmar2 жыл бұрын

    Awesome explanation

  • @somanshbudhwar
    @somanshbudhwar4 ай бұрын

    Epic!

  • @RohanShetty1992
    @RohanShetty1992Ай бұрын

    great content , thank you

  • @vineethp3644
    @vineethp36443 жыл бұрын

    Great explanation

  • @prathamesh5565
    @prathamesh55654 жыл бұрын

    Awesome 👍

  • @tendies
    @tendies3 жыл бұрын

    Well done,thank you

  • @itaco8066
    @itaco80662 жыл бұрын

    Awesome

  • @shaikhyusufniaz759
    @shaikhyusufniaz75910 ай бұрын

    Gold !

  • @F1mus
    @F1mus4 жыл бұрын

    Really good video.

  • @Chris-bq4qd
    @Chris-bq4qd5 жыл бұрын

    Very helpful.

  • @chitthiaayeehai
    @chitthiaayeehai5 жыл бұрын

    Perfect.

  • @MuhammadNadeem-rc5bk
    @MuhammadNadeem-rc5bk4 жыл бұрын

    Good work

  • @aurojitbasak3249
    @aurojitbasak32495 жыл бұрын

    Thank you for your video

  • @mrunalshende1449
    @mrunalshende14492 жыл бұрын

    Thank you so much..

  • @rozz984
    @rozz9845 жыл бұрын

    Hi , Can you make video related to transaction management ,propagations ,isolations,2-phase commit, deadlocks in transactions..

  • @sivagurunathanparanitharan4714
    @sivagurunathanparanitharan47144 жыл бұрын

    🙏 You are the Best Teacher that I have ever seen.🥇, A classical method of approach = Superb indeed.👌.🎯Please Could you kindly teach us Flutter & Dart & Go lang & Scala too . Pleeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeese Sir..... 🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏

  • @DefogTech

    @DefogTech

    4 жыл бұрын

    Thank you so much for the kind words. Definitely plan to continue creating videos, and also covering more topics like Dart and Go

  • @rutujadolas5347
    @rutujadolas53472 жыл бұрын

    Very nice explanation. Thank you so much for the same. I had a few questions - 1. Which type(s) pf write operation(s) can a client application perform on the files. In other words, what does GFS supports in terms of writing into files? 2. Where in the file the writing can be done? 3. How much data can be written there and by how many applications at the same time and at the same location? 4. In what type of memory is the metadata stored on the master

  • @spider-bot

    @spider-bot

    Жыл бұрын

    Bit late but hopefully others find it useful: Regarding (1) & (2), GFS was primarily aimed at applications which involved appending data to a long-running file, with random location writes happening rarely (if at all). For (3), the limits would depend on the actual implementation - this was just a theoretical paper on GFS. But yes, it is meant to support concurrent writes. And regarding (4), the Master keeps a track of all the files on the GFS, a list of chunk handlers that make up each file, and the list of chunk servers that hold a particular chunk (there are multiple servers for redundancy) Additionally the master keeps a log of all the changes that were made (along with checkpoints), so that it can recover from crashes

  • @rahulsinghai3033
    @rahulsinghai30335 жыл бұрын

    I really like your videos and regular follower of your channel. Please publish a video on Cassandra architecture. If possible.

  • @DefogTech

    @DefogTech

    5 жыл бұрын

    Yes sir! Cassandra is similar to Google Bigtable. Will cover it soon.

  • @sureshchaudhari4465
    @sureshchaudhari44652 жыл бұрын

    waw u explained so smilply

  • @user-rg1ii5lo5f
    @user-rg1ii5lo5f5 ай бұрын

    Gr8

  • @ajeetstationajeet
    @ajeetstationajeetАй бұрын

    its architecture somewhat similar to napster ( I think napster was ahead of its time)

  • @MrPrince0277
    @MrPrince02772 жыл бұрын

    If each chunk has 3 replicas, then will there be a total of 4 such chunks ?

  • @siripanchadi6288
    @siripanchadi62884 жыл бұрын

    💙

  • @sharanyarai378
    @sharanyarai3785 жыл бұрын

    Thanks, can you also explain MapReduce please?

  • @DefogTech

    @DefogTech

    5 жыл бұрын

    Absolutely. That video is coming tomorrow!

  • @kunal4350
    @kunal43503 жыл бұрын

    Hey , request you to please make new video. Eagerly waiting for your new videos.

  • @movingtoweb
    @movingtoweb5 жыл бұрын

    Good one buddy👏

  • @DefogTech

    @DefogTech

    5 жыл бұрын

    Thank you buddy! Btw, Congrats on 40K, that was fast!

  • @TechPrimers

    @TechPrimers

    5 жыл бұрын

    Defog Tech thanks dude🙏🏻

  • @sanjaykantheti4002

    @sanjaykantheti4002

    5 жыл бұрын

    I m happy to see 2 good Java content creators at one place. Continue the legacy. :)

  • @TechPrimers

    @TechPrimers

    5 жыл бұрын

    Sanjay Kantheti 🤘

  • @speak_charlie
    @speak_charlie2 жыл бұрын

    👍🏿👍🏿👍🏿

  • @nemiltimbadia7134
    @nemiltimbadia71343 жыл бұрын

    Can you please do Dynamo paper

  • @prashantjha439
    @prashantjha4394 жыл бұрын

    Hey , why have you stopped uploading videos....let me know if you need any funding

  • @DefogTech

    @DefogTech

    4 жыл бұрын

    hey, thanks so much for reaching out.. just been busy with new job, and added laziness :( I am researching for NoSQL series, but will take time. Regarding funding, absolutely.. I love teaching and hope to someday do this fulltime. Might start KZread subscriptions for earning. Let me know your thoughts, is ~130 INR a month something everyone would go for, or should it be course based fees on a separate website?

  • @prashantjha439

    @prashantjha439

    4 жыл бұрын

    @@DefogTech , this channel is my first point of reference for any cs related topic , things that make your videos unique are the awesome slides , quality content & very crisp explanation. I myself being a software professional at Adobe frequently watch your videos, so yes youtube subscription seem to be a viable option. Hoping you to resume soon so that we can make best amidst the lockdown. :)

  • @meghaharlalka
    @meghaharlalka5 жыл бұрын

    is the master something like a zookeeper?

  • @DefogTech

    @DefogTech

    5 жыл бұрын

    No, it's a single instance backed by a secondary one unlike concensus based one like zookeeper

  • @martinharris4416
    @martinharris44162 жыл бұрын

    The chunk is **64KB** NOT MB

  • @DefogTech

    @DefogTech

    2 жыл бұрын

    It is 64MB. GFS is used mainly for large files with append-only structure. In the paper there is good trade-off of why they chose 64MB.

  • @user-zs7mu4oq8e

    @user-zs7mu4oq8e

    Ай бұрын

    MB

  • @sahenamaharjan6167
    @sahenamaharjan6167 Жыл бұрын

    Explained clearly. Thanks.