This is by far the best explanation of the MapReduce technique that I have come across. I especially like how the technique was explained with the least amount of technical jargon. This is truly an ELI5 definition for MapReduce. Good work!

@jessetanderson

8 жыл бұрын

+Subramanian Iyer Thanks!

An innovative idea to use a pack of cards to explain the concept. Getting fundamentals right with an example is great ! Thank you

Very well done - not too slow, yet very clear and well structured.

Jesse may you get all SUCCESS and BLESSINGS

really cool one. It is always nice to come back to the basics. Thanks for that one

Great explanation !! You Mapped the Complexity and Reduced it to Simplicity = MapReduce :)

Great presentation. The visualization makes it so much easier to understand.

and that's how you explain any technical concept. simple is beautiful!

That was very helpful Jesse. Thank you for sharing this!!

Really liked your way of presentation....."Simple" and "Informative". Thanks for sharing!!

Wonderful explanation ! Made it very simple to understand! Thanks a ton!

Wow.. You have made this look so simple and easy... Thanks a ton !!!

Great explanation! This is how a tutor should simplify the understanding! Thanks

Your explanation is majic! Well done

what a great effort, i am astonished by your teaching skills.we need teachers like you.Thanks for your best explanation .

Great explanation!! worth a bookmark. Thank you sir!

Just wow...very nicely explained

The explanation is wonderful.. You made me understand things easily.

Nice video explaining the Map Reduce Practically.

an ounce of example is better than a ton of precept! --Thanks, this was great!

Good to understand for a layman! So its quite crucial to identify the basis of the grouping i.e. the parameters based on which the data should be stored in each node. Is it possible to revisit that at a later stage?

Really good illustration.... really easy to understand for people as me that we are not computer experts.. thanks

best explanation of mapReduce. Thanks!

really nice video and explain the terms in a simple way...

Superb. Thank you Jesse Anderson

loved the idea. Now I understood how map reduce works. Thank you.

Great video with good explanation technique.

Great Explanation! Thanks!

Brilliant approach to teach the concept

The only one I watched which can clearly introduce mapreduce to newbie

Very useful explanation.

amazing explanation! I love it. Huge Thanks!

Thanks this really helped me for my exam !!

Good Explanation with simple example

Nice tutorial! Easy to understand

Thank You sir for such a wonderful explanation. :-)

Thats wonderful ..... you are a gret teacher

Great summary - thanks!

Excellent explanation!

just great explanation !

Great explanation. Thanks.

Thanks! Great explanation

Good illustration using a practical example...

Best explanation. Thanks a lot

Great explanation!

Thank u very much for the explanation.

Excellent video explanation

Thanks for the great video!

Great lesson. Thanks..

Brilliant - thanks!

thanks! that is an easy explanation!

To wrap this up: Map = Split data Reduce = Perform calculations on small chunks of data in parallel Then combine the subresults from each reduced-chunk. Is that correct?

@jessetanderson

8 жыл бұрын

+mmuuuuhh Somewhat correct. I'd suggest buying the screencast to learn more about the code and how it works.

@alphacat03

8 жыл бұрын

+mmuuuuhh merge-sort maybe?

@kemchobhenchod

7 жыл бұрын

divide and conquer

@BULLSHXTYT

6 жыл бұрын

Map transforms data too

@dennycrane2938

5 жыл бұрын

No no... Map = Reduce the Data, Reduce = Map the Data . .... ....

Now I get it, thanks!

Never trust a man whose deck of playing cards has two 7s of Diamonds.

Superb video....thanks a lot sir

great video by the way!!

It was very nice. But I could not find the video that you showed the shuffling "magic part"

So it follows mainly the principle of divide and conquer?

@jessetanderson

6 жыл бұрын

Following that analogy, it would be divide, reassemble, and conquer.

Good illustration. 😃

Very nice, thanks a lot.

Brilliant!

If I understand correctly, the mapper divvies up the data among nodes of the cluster and subsequently organizes the data on each node into key-value pairs, and the reducer collates the key-value pairs and distributes the pairs among the nodes.

@jessetanderson

2 жыл бұрын

Almost. Hadoop divvies up the data, the mapper creates key value pairs, and the reducer processes the collated pairs.

Thanks Jesse! This is a wonderful video! I have 2 doubts. 1. Instead of sum, if it is a sort function, how will splitting it into nodes work? Because then every data point should be treated in one go. 2. The last part on scaling, how will different nodes working on a file and then combining based on key, be more efficient than one node working on one file? I am new to this and would appreciate some guidance and help on the same.

@jessetanderson

5 жыл бұрын

1. This example goes more into sorting github.com/eljefe6a/CardSecondarySort 2. It isn't more efficient, but more scalable.

@AnirudhJas

5 жыл бұрын

@@jessetanderson Thank you!

Well, that explains the interview question: How would you sort a ridiculously large amount of data?

When you say nodes and clusters, does an input file of 1TB should definitely be run in more than one computer or we can install hadoop in a single laptop and virtually create nodes and clusters ?

I actually did this with cards.Thanks

dude, whats the name of that magic??

i like this technique nice keep it up

Good explanation

Great video

Hat's of man. very well understood

6:16 got a question! Would you please elaborate more on those moving data? Since there is two separate reduce task on those two nodes how does two different reduce tasks combine together? How do we choose which cards move to which node?

@jessetanderson

10 жыл бұрын

That is called the shuffle sort. See more about that here www.inkling.com/read/hadoop-definitive-guide-tom-white-3rd/chapter-6/shuffle-and-sort.

@chandrakanthpadi

3 жыл бұрын

Does the actual data in the node moves or copies of the data is moved?

awesome

greta video. why is there performance issues with hadoop however?

@jessetanderson

9 жыл бұрын

I'm not sure what you mean by performance issues.

wow this was great

The magic part u mentioned in the video resides in reducer or Map?

@jessetanderson

6 жыл бұрын

Shoaib Khan mostly in between those two phases

Aiwa. Simply explained.

Wonderful, you have used the right tool(cards) and made it simpler. Thank you. Am i correct in saying, in this manual shuffle and sort, the block size is 52 cards where as in a node it would be 128.

Great video, thanks for sharing !

awesome explanation super

excellent!!!

Hi Jesse, can I use map reduce only on document-oriented DBs, or also e.g. on Graph databases?

@jessetanderson

6 жыл бұрын

Hessebub you can use it for both, but the processing Algorithms are very different between them.

@paulfsch3108

6 жыл бұрын

Alright, thanks very much for answering & doing the video in the first place!

4:51 - - i'm kind of lost. so you said two papers as two sets of nodes. left is node1 and right is node2. then you said, "I have two nodes, where each node has 4 stacks of cards". I also understood that you are merging two varieties of cards in node1 and another two varieties of cards in node2. " a cluster is made of tens, hundreds or even thousands of nodes all connected by a network". so in this example, let's say two papers(nodes) are one cluster. the part I get confused is , when you say " the mapper on a node operates on that smaller part. the magic takes the mapper data from every node and brings it together on nodes all around the cluster. the reducer runs a node and knows it has access to everything with same key ". So if there are two nodes A and B that has mapper data, then the reducer part will happen on two other nodes C and D. I'm confused when you say "on nodes all around the cluster".

Which music is this in start of this video

@jessetanderson

3 жыл бұрын

I'm not sure where they got it from.

Might be a bit clearer to understand the advantage of this if instead of having the same person run the cards on each node sequentially and have two people do it at the same time. Or go further and have four people show it. Then each person can grab all the cards of the suit from each node and can sum their values up, again, at the same time. Show a timer showing how long it took for the one person to do everything on one node and the time of having all four running at the same time.

thanks

What if the node with clubs and hearts breaks down during the reduce operation? Will data be lost? Or will the complete Map Reduce job be repeated using the replicated data?

@jessetanderson

Жыл бұрын

The data is replicated and the reduce would be re-run on a different node.

spade clubs ... I think you used the wrong suite names for them :)

awesome

so nice

huge 1Tb file.. anyone watching this in 2065?

@NuEnque

5 жыл бұрын

February 2019 (Go RAMS)

@anmjubaer

5 жыл бұрын

@@NuEnque July 21 2019

@devanshsrivastava4265

4 жыл бұрын

feb 2020

@jonathannimmo9293

4 жыл бұрын

more like 2025

@omrajpurkar

4 жыл бұрын

August 11, 2020!!

Great

my friend: i wish i had ur calm we having an exam tomorrow you watching how playing cards....

Cool

little bit long explanation. could be done faster (e.g. card-sorting). But after watching, you know what's happening. So all thumbs up!

Easiest explanation.

Interesting. Now I want to request a bunny comes out from a hat.

The 'scalability' of hadoop has to do with the fact that the data being processed CAN be broken up and processed in parallel in chunks and then the results can be tallied by key. It's not an inherent ability of the tech other than HDFS itself. Like most technology or jobs for that matter the actual 'process' is simple it's wading through the industry specific terminology that has makes it unnecessarily complicated. Hell you can make boiling an egg or making toast complicated too if that's your intent.

@jessetanderson

5 жыл бұрын

Sorry, you misunderstood.

@ZFlyingVLover

5 жыл бұрын

@@jessetanderson I didn't misunderstand you. Your explanation was great.

This is just a sales pitch

@jessetanderson

7 жыл бұрын

I think the description is pretty clear that it's an extended preview of the screencast.

This is a great example video without the accent to deal with.