How NETFLIX onboards new content: Video Processing at scale 🎥

Everyday, #Netflix handles billions of requests regarding movies, trailers and other video content. Delivering at such a large scale needs an #engineering marvel. This #video will talk about how Netflix is able to onboard new video content onto their platform. We go from video chunking to collating 4 second shots into scenes.
Amazon S3 is used to store the video chunks. Netflix also provides open connect servers to internet service providers, which acts like a cache of movies. Most requests to Netflix can be served by this cache, and the remaining are sent over the network. This reduces the bandwidth and time required for Netflix to operate at scale. Synergy at it's finest.
00:00 Problem Description
00:32 Video formats and resolutions
03:18 Chunk processing
05:52 Storage
06:19 OpenConnect for video caching
10:13 Summary
Looking to ace your next interview? Try this System Design video course! 🔥
interviewready.io
With video lectures, architecture diagrams, capacity planning, API contracts and evaluation tests. It's a complete package.
References:
/ high-quality-video-enc...
/ optimized-shot-based-e...
/ video
/ stream
/ keystone-real-time-str...
media.netflix.com/en/company-...
System Design Playlist: • System Design for Begi...
You can follow me on:
LinkedIn: / gaurav-sen-56b6a941
Instagram: / applepie404
Quora: www.quora.com/profile/Gaurav-...
Twitter: / gkcs_
External content used:
Fast and Furious 7 (Car chase scene)
Boone Loves Video: • 🌎Create 3D PATH ANIMAT...
Sacred Games (Available on Netflix)
Johnny English Reborn (What's it called?)
Netflix Logo

Пікірлер: 540

  • @adityatripathi1904
    @adityatripathi1904 Жыл бұрын

    I believe there is a variation to Gaurav's understanding on open-connect caching mechanism. A new uploaded video DOES NOT goes directly to the caching server, instead they are resolved to a 'a small Netflix DB replica" in that region, which in-term populates the caching layer for new videos or most frequently accessed videos. The distinction is important because otherwise the design is assumed to directly populate OC CDNs, which is not the case. So for example, if a viewer in Australia wants to stream a non-cached US content: Instead of directing all the Netflix traffic through under-sea cables, Netflix copies the video file (transcoded) from US-based storage to a storage location in Australia once during off-peak hours. Once the video has reached the continent, it’s copied to all the Open Connect servers present in the ISP networks. This is not to point out a flaw in the video, just a clarification.

  • @gkcs

    @gkcs

    Жыл бұрын

    Thank you for this amazing clarification 😁

  • @adityatripathi1904

    @adityatripathi1904

    Жыл бұрын

    @@gkcs I am glad. Keep making fun content😄

  • @Nikhilajayk
    @Nikhilajayk4 жыл бұрын

    Content apart , the new animations that you've added are really nice. Extra efforts are appreciated! Keep em coming !

  • @nehaambasta1191
    @nehaambasta11914 жыл бұрын

    Man, you know how to teach with fun! I must say presentation and video editing is very well done in your videos.

  • @manishramesh5446
    @manishramesh54464 жыл бұрын

    Wow, excellent video. Love your unique style of breaking down the topics in chunks and explaining them neatly. Keep them coming.

  • @akashthoriya
    @akashthoriya4 жыл бұрын

    holy shit! Just fall for these videos; Thank you from bottom of my heart! Please make more video on this topic!

  • @adityapatange8
    @adityapatange84 жыл бұрын

    What a beast! Loved your reference to Amazon S3 out of nowhere! 😎✔️

  • 2 жыл бұрын

    The material you have in this playlist is gold! I did not expect to learn so much from it. Thanks a lot!

  • @gkcs

    @gkcs

    2 жыл бұрын

    You're welcome!

  • @himanikumar7979
    @himanikumar79793 жыл бұрын

    Blown by the simplicity you explained such a complex topic!

  • @amarchhabra2175
    @amarchhabra21754 жыл бұрын

    Bro - Thanks for the great video! Learned a lot. Just subscribed. Keep the great content coming!

  • @walaaelsayed8764
    @walaaelsayed87644 жыл бұрын

    Very helpful and simple for anyone who is preparing for software design interview. Your way of explaining is interesting and funny 😊

  • @sushruttabakade6088
    @sushruttabakade60884 жыл бұрын

    You have the most unique content on KZread, man. No bs, pure knowledge. Keep going.

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks!

  • @algoseekee
    @algoseekee4 жыл бұрын

    It's amazing to see how Guarav improves the quality of his content, and I can tell you, guys, as a newbie tech KZreadr, it's a big deal. Keep it up, bro! ;-)

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks Victor!

  • @ShivamRaj-si5wt
    @ShivamRaj-si5wt3 жыл бұрын

    1:23 AWESOME GAURAV, perfect example of what you are explaining, Huge respect bro

  • @sumankumarsubudhi
    @sumankumarsubudhi4 жыл бұрын

    Whoa!!! Thank you so much for making such unique and great content videos ;)

  • @uneq9589
    @uneq95894 жыл бұрын

    As i have some experience in this area, i would like to say that you delivered crisp content on the same. Good job!

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thank you!

  • @ayasswain
    @ayasswain4 жыл бұрын

    Another informative video from you. Nice explanation. Thanks. Just wanted to add something on the caching part. Many websites take help of CDN providers like Akamai to do the caching on behalf of them. The CDN providers have the required infrastructure across the globe wherein they have placed their caching servers in most of the countries.

  • @gkcs

    @gkcs

    4 жыл бұрын

    That's interesting, thanks!

  • @syjaf6615
    @syjaf66154 жыл бұрын

    Awesome man! Simple and to the point. The best kind of online content.

  • @mrskwrl
    @mrskwrl4 жыл бұрын

    Love the energy. Your videos are great.

  • @BhageshDotIn
    @BhageshDotIn4 жыл бұрын

    Hi Gaurav, thanks for the video. Impeccable. Keep doing the great work!!

  • @INCREDIBLEGUYOFINDIA
    @INCREDIBLEGUYOFINDIA4 жыл бұрын

    Have been working in this arena for a while now, you got everything correct man, other stuff you post usually goes over my head because I haven't dabbled in a lot of those things but for once, felt nice to already know what you were gonna say. Haha.

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks! Your replies on the comments are excellent btw :D

  • @shubhankar915
    @shubhankar9154 жыл бұрын

    Showing Gupt in local popular movie would make Booby Deol really Proud. Savage AF Gaurav! :D

  • @vishnuthamizharasan
    @vishnuthamizharasan4 жыл бұрын

    To be honest learned something useful today, and willing to learn from you more like this kinda of topic!

  • @tirthdoshi7463
    @tirthdoshi74634 жыл бұрын

    Gaurav, I can't tell how interesting this video was ! Thanks a lot ! I totally enjoyed learning this ! Keep making more such videos ! :)

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks!

  • @pradipkumarkarnavat1205
    @pradipkumarkarnavat12054 жыл бұрын

    I was waiting for this video, awesome

  • @PyjamaLama
    @PyjamaLama4 жыл бұрын

    Wow this is sweet! Super smart solution to the issues of video streaming, will be interesting to see where it is improved upon in the future too!

  • @andriidanylov9453
    @andriidanylov9453 Жыл бұрын

    Thank You for sharing. Very nice details and very good description

  • @kennethpchang
    @kennethpchang4 жыл бұрын

    I'm very impressed! Great explanation! I just subscribed! Keep going!

  • @KomalSingh-bh8zr
    @KomalSingh-bh8zr4 жыл бұрын

    Thanks for this wonderful insight on the engineering side of Netflix, looking forward to more system design videos.

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks!

  • @neeshespieces
    @neeshespieces3 жыл бұрын

    You’re awesome. Thanks 😁😍 this was so helpful

  • @shraddhapandey5442
    @shraddhapandey5442 Жыл бұрын

    That was a nice one ! Thanks for the video.

  • @sankalparora9374
    @sankalparora9374 Жыл бұрын

    Short, simple and full of interesting concepts. Thanks!

  • @gkcs

    @gkcs

    Жыл бұрын

    Thank you!

  • @prnk139
    @prnk1392 жыл бұрын

    great efforts thanks a lot ! i remember my childhood days after seeing the GUPT poster! :-)

  • @AnikBanerjeeliveforfun
    @AnikBanerjeeliveforfun4 жыл бұрын

    Moral : Steal that Open Connect Box

  • @videoguy640
    @videoguy6403 жыл бұрын

    That bit at the beginning was hilarious 😂

  • @user-zu2uh8zg6p
    @user-zu2uh8zg6p7 күн бұрын

    Thanks for your descriptions

  • @gkcs

    @gkcs

    5 күн бұрын

    Cheers!

  • @mrinalinimusic
    @mrinalinimusic4 жыл бұрын

    Your videos are pure delight to watch !

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thank you!

  • @shubhamtech89
    @shubhamtech892 жыл бұрын

    Champion presentation. Just wanted to see a graphical representation of the adaptive bit rate ladder through Netflix's eyes.

  • @Rishabh199494
    @Rishabh1994942 жыл бұрын

    I just started watching your videos. Your content is Amazing.

  • @amitjain-eth
    @amitjain-eth3 жыл бұрын

    Nice and depth knowledge bro!

  • @MukundPadale
    @MukundPadale4 жыл бұрын

    Awesome bro ..!! Simple and very informative.. loved it .. keep it up 👍

  • @tejassardana6266
    @tejassardana62664 жыл бұрын

    Don't be surprised if you hit 500K subscribers by the end of 2019. Your content quality is skyrocketing.

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thank you!

  • @DiaryOfMuhib

    @DiaryOfMuhib

    3 жыл бұрын

    @@talesara74 It's system design and it's not for everyone to understand, what did you expect? People mainly use this channel for technical system design interviews.

  • @pratsway961

    @pratsway961

    2 жыл бұрын

    You said that from your perspective. But the sad reality is People really don't like good contents😅😂. They can watch a 3hrs movie, but couldn't watch this type videos for 10mins😑

  • @letsrock7354

    @letsrock7354

    Жыл бұрын

    Gaurav I love your content and soon I guess will be joining your InterviewReady platform...Please offer me some special discount bhai 😍

  • @vinayak186f3

    @vinayak186f3

    Жыл бұрын

    It's almost 2023 and he still has 488k 🥲

  • @SuboptimalEng
    @SuboptimalEng4 жыл бұрын

    Very detailed. Great work!

  • @toabhijeetsingh
    @toabhijeetsingh4 жыл бұрын

    The content of this video was very dense. The breather at 5:29 was very much wanted :-). You have very good audience engaging skills. Good Luck !!

  • @muthaiahpalaniappan8009
    @muthaiahpalaniappan80093 жыл бұрын

    As always one another brilliant content

  • @anupammangal8990
    @anupammangal89903 жыл бұрын

    Hey Gaurav, I was just wondering how the Open Connect or KZread Red boxes are any different from the long existing concept of CDN (Content Delivery Network). The only change is, being big companies they have the money and resources to create their own personal CDNs and brand them which is kind of building your own house instead of renting one. Is there something I am missing here ?

  • @gkcs

    @gkcs

    3 жыл бұрын

    They are very similar, yes.

  • @samiles171094

    @samiles171094

    2 жыл бұрын

    I was having the same question as how these Oppen connect boxes are different from CDNs.

  • @siddheshmadiwale4352
    @siddheshmadiwale43524 жыл бұрын

    that's pretty interesting concept... very well explained 👏👏.....

  • @pvrcs
    @pvrcs4 жыл бұрын

    Can you explain some of the advanced concepts of video streaming too? For eg., Bandersnatch? I wonder how Netflix is providing conditional streaming of video chunks based on user selection.

  • @lala7701
    @lala77014 жыл бұрын

    I just love your system design videos.

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks 😁

  • @yashgandhi9698
    @yashgandhi96984 жыл бұрын

    I was the first one to watch this video after him! The content had no views but a like!!

  • @Lokesh_Karki

    @Lokesh_Karki

    4 жыл бұрын

    That's a improper system design for KZread, probably slow.

  • @Iamjohnsmithindia
    @Iamjohnsmithindia4 жыл бұрын

    Keywords = CDN , predictive content delivery , content pre-fetch and so on...

  • @vinayshrivastava266

    @vinayshrivastava266

    4 жыл бұрын

    Yes Content Delivery Network plays a really important role if the geographic location of physical server is far and also I think what these OTTs would be doing is what we are getting in recommendation might be based on geo location rather than your watch history , this way if one guy from ur location has seen some movie it gets cached to a CDN and then when u request for the same content , since it comes in ur recommendation its extremely fast

  • @kajalpareek8291
    @kajalpareek82914 жыл бұрын

    Very nice explained and made it interesting. Will be waiting for more good system design question.

  • @ankitbagga5713
    @ankitbagga57134 жыл бұрын

    Thanks Gaurav. I am not an engineer but always wanted to get some resource to learn the concepts. I would love to see more of your videos and on advance concepts.

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thanks Ankit!

  • @meghakashyap1303
    @meghakashyap13034 жыл бұрын

    Thank you Gaurav for the content . Hope to see many more useful content .

  • @CodeMode9313
    @CodeMode9313 Жыл бұрын

    Paaji ek dum kamaal ka video hai ...even understoodable for non cs

  • @djlyons90
    @djlyons902 жыл бұрын

    This was a very informative video! Awesome job.

  • @kennethcarvalho3684
    @kennethcarvalho3684 Жыл бұрын

    You are God of System Design!!

  • @logicboard7746
    @logicboard77462 жыл бұрын

    Thank you for your reading on this topic...this is enlightening (at least for me)

  • @anushreejain98
    @anushreejain984 жыл бұрын

    must say u made it really Interesting with editing :P and great content!

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thank you!

  • @khalilladha
    @khalilladha4 жыл бұрын

    Great, Nicely explained !!!

  • @vk21563
    @vk215634 жыл бұрын

    ISP cache is really interesting, wondering how they have handled security on top of it across multiple geography??

  • @dicksonrobert1492
    @dicksonrobert14924 жыл бұрын

    Thank you for the explanation

  • @sudiptdabral5590
    @sudiptdabral55904 жыл бұрын

    Great content always!

  • @saadabdulhameed6699
    @saadabdulhameed66994 жыл бұрын

    That's just awesome Keep it up

  • @debagnikroy9450
    @debagnikroy94504 жыл бұрын

    OMG!!!! the opening😂😂

  • @amraboeleneen2221
    @amraboeleneen22214 жыл бұрын

    real effort here, good job mate

  • @manirajsivasubbu4623
    @manirajsivasubbu46233 жыл бұрын

    Nice explaination, Thank you so much.

  • @luis96xd
    @luis96xd3 жыл бұрын

    Excellent video, well explained!

  • @aadeshsharma0001
    @aadeshsharma00012 жыл бұрын

    this isp cache thing is just simply amazing,

  • @arsalalam5998
    @arsalalam59984 жыл бұрын

    Brief and precise.. Nice.

  • @rohscx
    @rohscx4 жыл бұрын

    Good job explaining!!

  • @vibekdutta6539
    @vibekdutta65394 жыл бұрын

    Happy Teachers Day Sir! U taught me a lot! I really appreciate that!

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thank you!

  • @galfas09
    @galfas094 жыл бұрын

    Great video ! Thanks for sharing

  • @anjalibelani5101
    @anjalibelani51013 жыл бұрын

    This is great work Gaurav. Thank you for this!

  • @gkcs

    @gkcs

    3 жыл бұрын

    😁

  • @dhruvenvora
    @dhruvenvora2 жыл бұрын

    Great breakdown!

  • @shoaibzafar5663
    @shoaibzafar5663 Жыл бұрын

    Outstanding 😨😱

  • @jayeshmali9240
    @jayeshmali92402 жыл бұрын

    Great presentation skills and content🔥🔥🔥

  • @gouravraghuwanshi
    @gouravraghuwanshi4 жыл бұрын

    Superb explanation bro

  • @scenescentral3004
    @scenescentral30042 жыл бұрын

    Fantastic explanation

  • @Theartsygalslays
    @Theartsygalslays2 жыл бұрын

    Very well explained, thank you, Gaurav!

  • @gkcs

    @gkcs

    2 жыл бұрын

    Thanks Veronica 😁

  • @vikramrajput76
    @vikramrajput764 жыл бұрын

    Really helpful man!

  • @yog2915
    @yog29152 жыл бұрын

    Bro u are great am a 14+ software engineer really appreciate ur content

  • @charanteja4632
    @charanteja46324 жыл бұрын

    Please do some lectures on docker internals like Cgroups, namespaces, union capable file systems

  • @prankurverma8967
    @prankurverma89674 жыл бұрын

    So well explained 🤟

  • @Richard_GIS
    @Richard_GIS4 жыл бұрын

    Awesome!!

  • @sumitlahiri209
    @sumitlahiri2094 жыл бұрын

    Awesome Video. Very good explanation. Video content caching is not an easy job. The video is really good.

  • @sumitlahiri209

    @sumitlahiri209

    4 жыл бұрын

    Software Architecture is the most important thing today. No matter how much you you improve on ML algos or buy hardware etc... It is the archtecture that defines quality and reliability. How you connect components together using a good deisgn tactic and make each component work flawlessly by the deisgn pattern matters the most. It is archtecture everywhere. The fact that you make videos on it is really a good thing. Developers today must understand the imporatnce of it. It is not just writing code but many other things too.You are doing a great job by making such videos.

  • @inderdhami2001
    @inderdhami2001 Жыл бұрын

    Very informative!

  • @shivnandantiwari7489
    @shivnandantiwari74893 жыл бұрын

    You're a gem you should have a teaching startup like byjus

  • @Cricketming
    @Cricketming4 жыл бұрын

    Great content.. keep up the good work..

  • @SA-bp5kw
    @SA-bp5kw4 жыл бұрын

    Not a programmer or anything. But I love your videos. It's interesting.

  • @deepakpai0796
    @deepakpai07964 жыл бұрын

    Thats a really great explanation. Thanks

  • @gkcs

    @gkcs

    4 жыл бұрын

    Thank you!

  • @mdaamirkhan5901
    @mdaamirkhan59014 жыл бұрын

    Nice work.

  • @sarvarthmonga5764
    @sarvarthmonga57642 жыл бұрын

    Gupt yrr, haha Very wonderfully explained Thanks for that

  • @amitupadhyay6511
    @amitupadhyay65112 жыл бұрын

    Not gonna lie, you got me in first second, sorting in O(n) lol

  • @Live-hh6li
    @Live-hh6li4 жыл бұрын

    Awesome Video 😍

  • @manjunath2728
    @manjunath27284 жыл бұрын

    Hi Gaurav Thanks for awesome video. What are the technologies Netflix uses to display the videos?.

  • @ruchirai5775
    @ruchirai57753 жыл бұрын

    :) Thanks this is amazing

  • @ashishrana7519
    @ashishrana75194 жыл бұрын

    Bhai saab.. Creative content...

  • @GauravJain108
    @GauravJain1084 жыл бұрын

    Thank you!

  • @anujpachawadkar4948
    @anujpachawadkar49484 жыл бұрын

    What about user authentication and authorisation when dealing with open connect??

  • @kashyapsreview
    @kashyapsreview3 жыл бұрын

    Thanks for the great info I wonder why I like Netflix more than any other OTT