I've looked through various resources for system design and this is the first time I've heard of request collapsing. Really appreciate the unique content on your channel. It's nice to see deep dives of actual system architectures instead of bottom up conceptual simplified architectures for these problems. In a similar vein, I really enjoyed your Netflix video too. Thanks for all the great content!

@JitendraSarswat

4 жыл бұрын

Yes, that's actually a great concept. We use hystrix for the same purpose. Hystrix is generally use to make microservices fault tolerant. But request collapsing is another use case for the same, if your architecture happen to use Java along with Spring framework, you can use the mentioned tool to implement the same.

@SubhramRana

Жыл бұрын

This is very similar to "thundering herd" problem, see the video "how facebook scales memcache"... This problem is solved uaing "leasing" technique

You really have my respect Naren . Thanks a lot for sharing your knowledge with us in such a beautiful way .

Very nice. It's great to see so many new concepts which I had no clue about earlier. Thanks a lot man

I think keeping a scheduler which keeps polling on product APIs might increase load on product API and core API servers because it polls 2-3 times a second and there can be multiple sports events running at the same time and at this scale, only a single scheduler won't work, we need to keep multiple such schedulers, which will further increase load. Also, 2-3 times a sec. might not be that real time but that depends on the data it is polling for. What we can do is, we can publish the score updates to a queue like RabbitMQ and have a separate service which consumes this queue (basically pub-sub model) and writes it to database and we keep another service just for real time score updates, which also subscribes this queue and handle client connections for realtime score updates. It'll just listen to the queue and fanout the score updates in realtime over web sockets. We can also scale this by keeping multiple such services to handle huge number of clients.

Really would like to see design pattern for: 1. App store/ Google play 2. Elevator system 3. Chess

Just awesome...one of the best system design concepts I have come across

Awesome video did not know about 3 tier caching and varnish. Thank you for making these videos.

Great and Thanks a lot . One of the best channel to learn design related problems.

Very good explanation of CricBuzz system design

Keep up the good work, Narendra!

Thank you for sharing the information .. really appreciate your work

Its really great initiative dude nice explainations

This is pure gold. Thank you!

Thank you so much for your videos! Very helpful.

Great video . It gives an idea how several open source libraries can be leveraged . It clears lot of doubts also . Request you to create a video for stock trading system. It should include dynamic price update of shares & support rollback .

This video is awesome for System design which covers a lot of concepts. Thanks a lot Naren for educating people free of cost despite your busy days due to office works. Can I request you to make videos on following:- 1) How does Google search engine works(so much, so quick) 2) System design of any E-com websites like Amazon, Flipkart 3) System design of IRCTC or any booking website 4) Whatsapp System design

@TechDummiesNarendraL

5 жыл бұрын

Whatsapp system design video already available, i will consider the rest. Thanks

This is really helpful. Thanks.

Its really great to see your video and understanding the logical thinking behind while system designing. A genuine request if you can share multimodal travel or air ticket booking something related system design would be really great. Thank you in advance.

Too many features to talk in a single video. Ideally the live score update could have been an interesting feature and a deep dive into that would have been great.

Why use Queues to store notifications, I guess storage is only required when user is offline, in case of online, there will be regular updates through websocket, so no need for queue storage, will it not be a bad User Experience to send multiple notifications, instead we should try sending the latest notification when the user comes back online.

Hi Narendra, a quick question, how do cricbuzz create queues dynamically( per match) and also why use redis queues compared to activemq / rabbitmq etc . I have heard of temprorary queues in activemq ,is this something that we can use here.

Thanks! This is really helpful. How do you get the information?

Is the mixer/binder a good use case for graphQL? The resolvers can decide which micro-service needs to be called.

This guy is celebrity in system design

Please increase the volume please. Great video btw!

How do you cache using the product api? I know about memcache/redis. To a certain extent, batch requesting. Do you mean caching using nginx?

I don't understand why so much product description rather than covering core feature i.e. how to display scoreboard, which is a main problem. I don't think anyone going to ask how cricinfo make its microservice architecture fault tolerant and how the use hystrix. I request you to keep focus on core feature of any system.

@nitinkulkarni7942

4 жыл бұрын

I agree, This is a very rare video from Tech Dummies which is disappointing

Thank you for the video!

In many Amazon interviews, they usually ask Command execution framework design and play store design. Can i have material for that?

Is the delta in fast cast calculating the diff for each user?

At 21:33 , Varnish cache was so nicely used. Also, is cache TTL set to 1s coz there would be mostly queries for real time data and stale data (data of more than a second ago) is rarely queried for? As usual, amazing content , Loved every bit of it❤

Great content from Naren as always. Thank you for this. Here are a few questions I had. 1. 9:48 - There are public product APIs which are mostly JSON based for input and internally calling Core APIs through mixers/binders. Can GraphQL be an example of the same. 2. 17:06 - Varnish is used between the Product APIs and the Core APIs to allow for request collapsing to avoid stampede effect. 25:15 you have Hysterix also doing something similar ( in addition to circuit breaking to 500 and default response sending which might also be needed for the core APIs ), so why was Hysterix itself not used for the Core APIs as well instead of varnish? 3. 17:04 - How do we decide when to put data into L2 cache which is distributed? Also since we are using Redis as a queue, can we also use Redis as a distributed in memory cache. Are we putting stuff to the distributed cache when we are evicting it from the local product cache? 4. 30:32 - A scheduler is used basically to ensure that internally we are using the same pull APIs itself right? Instead can we have an architecture, where the write APIs ( which happen when the commentrator and the score board are doing ) add it to the scheduler which is a queue. I see that would lead to a lot of logic repetition but would reduce the load on the product APIs right? Again this is based on the load distribution between push and pull clients. Also instead of Delta, can we have the scheduler use the E-Tag and the If-Modified-Since/If-None-Match params there to ensure that it is putting something to the queue only when there is a modification? 2 and 3 come from a POV of reducing the number of different tech stacks :)

Ur are doing great job keep it up... And can u explain db design for circbuzz i was asked this question in one of d start-up. Thanks in advance

Another Awsome video by Tech Dummies!!! Can you suggest some book or web resources which I can refer for architecture and design stuff?

@TechDummiesNarendraL

5 жыл бұрын

Thanks @Rais More than any book try to follow tech blogs of famous tech giants eg: code.fb.com/ medium.com/airbnb-engineering blog.twitter.com/engineering/en_us.html developers.googleblog.com/ highscalability.com/ eng.uber.com yahooeng.tumblr.com/

Thank you Bro great video.. Request you to make some video on Payment system.

There is a little typo *design (both the places), btw great video.

Please make a video on Online chess game design

Your voice like Harsha Bhogle ❤️

For the notifications use case, you may have 100s of millions of clients waiting for notifications for a big match. How would you efficiently maintain that many web sockets? Are there any considerations?

this is great!!

I want to remove that string is hanging from his cap. this information is pretty useful, thank you

Please make a video on Log Monitoring System

Hysterix - Circuit Breaker needs to be explained in better way - it isolates nonresponding microservices , tries to return default response or replaces those non working microservices with default one to avoid cascading effect .

your videos are really very good , Ihave learned a lot. Can you please add a video for system design for Gaana/spotify/music streaming apps?

Hi Narendra, can you please share all your tech specific bookmarks

should we only save the most frequent request on cache? any algorithms for identifying such resources?

@juggernaut420

Жыл бұрын

LRU?

Hey Naren, Thank you so much for creating awesome contents on system design. I have a request, can you please create system design videos on: * Zomato * KZread * Amazon * Tinder Thank you

@TechDummiesNarendraL

5 жыл бұрын

Hi @Saurav sure, this actually helps. Thanks

How do you know all these system design?

Hi, You are video are great. If possible, can you add systemdesign for TradingView

Is varnish a standard term

Why you have used NoSql DB?

Please make a video on how hotstar managers to show stream to 20M users

wanted to understand the rationale behind the choice of db , the schema we want to use in DB , sharding strategy , queue which has topics about ongoing matches ..saw something about product apis core api's..totally out of context

Thanks a lot Naren for your efforts in making us understand system design concepts Can we have video on below topic (it is trending these days due to IPL 2020). 1.) Design Dream 11 (Fantasy League, Leaderboards etc.,)

THIS COMMENT IS FOR MY PERSONAL REFERENCE. TO UNDERSTAND PROPERLY WATCH THE FULL VIDEO -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- push vs pull 2:13 get initial score into system 3:35 nature of the system 7:41 HLD 8:09 API strategy 8:15 9:13 9:41 11:00 partial failure handle 14:22 caching 16:01 why local cache 17:00 stamped effect 19:01 circuit breaker 23:07 API monitoring 27:50 Broadcast fast cast 30:01

can you do a system design for paypal?

@TechDummiesNarendraL

5 жыл бұрын

Sure

@TechDummiesNarendraL

5 жыл бұрын

@kai13man Sure

Can you tell system design for Netflix???

@TechDummiesNarendraL

5 жыл бұрын

It's already available at my channel, do subscribe

@arjunanselvam

5 жыл бұрын

@@TechDummiesNarendraL Thanks Bro. Awesome work. Keep posting :)

bhai. Respect _/\_

Lol. The 29% traffic from US must be all Indians.

please make video on red bus and vlc media player

CMS

Just subscribed. Great content. Thank you. ✌💹💯