What is Database Sharding?

Ғылым және технология

🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
► LinkedIn: / anton-putra
► Twitter/X: / antonvputra
► GitHub: github.com/antonputra
► Email: me@antonputra.com
▬▬▬▬▬▬ Related videos 👨‍🏫 ▬▬▬▬▬▬
👉 [Playlist] Kubernetes Tutorials: • Kubernetes Tutorials
👉 [Playlist] Terraform Tutorials: • Terraform Tutorials fo...
👉 [Playlist] Network Tutorials: • Network Tutorials
👉 [Playlist] Apache Kafka Tutorials: • Apache Kafka Tutorials
👉 [Playlist] Performance Benchmarks: • Performance Benchmarks
👉 [Playlist] Database Tutorials: • Database Tutorials
▬▬▬▬▬▬▬ Timestamps ⏰ ▬▬▬▬▬▬▬
0:00 What is database sharding?
0:12 Why is database sharding important?
0:58 What are the benefits of database sharding?
1:57 How does database sharding work?
3:42 What are the methods of database sharding?
3:47 Range-based sharding
4:59 Hashed sharding
5:52 Directory sharding
6:38 Geo sharding
7:25 How to optimize database sharding for even data distribution?
7:45 Cardinality
8:05 Frequency
8:24 Monotonic change
▬▬▬▬▬▬▬ Source Code 📚 ▬▬▬▬▬▬▬
► GitHub: github.com/antonputra/tutorials
#kubernetes #devops #cloud

Пікірлер: 80

  • @AntonPutra
    @AntonPutra11 ай бұрын

    🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com

  • @jasper5016
    @jasper50164 ай бұрын

    Hard to believe this top-notch content has very less views. Thanks a lot, Anton!!

  • @AntonPutra

    @AntonPutra

    4 ай бұрын

    ❤️

  • @jackfrost8969

    @jackfrost8969

    3 ай бұрын

    probably coz it basically has no volumn

  • @agun21st
    @agun21st11 ай бұрын

    Thank you sir for detail explaination of database sharding. We hope a practical handson of Database sharding will publish soon.

  • @PhillyHank
    @PhillyHank7 ай бұрын

    Very helpful. Very confused and to the point! I hope your colleagues who do technical videos would follow your framework. 👍🏾👍🏾👍🏾

  • @xardiannon5038
    @xardiannon50384 ай бұрын

    Superb explanation, and never strayed off topic.

  • @Xaoticex
    @Xaoticex5 ай бұрын

    Nice, exhaustive and short video considering it covers a lot.

  • @ashutoshtiwari4398
    @ashutoshtiwari4398Ай бұрын

    No Nonsense, direct to point, covering all cases. Well-compiled video!

  • @AntonPutra

    @AntonPutra

    Ай бұрын

    thank you!

  • @GabrielPozo
    @GabrielPozo11 ай бұрын

    Great video! Always an important topic when we think about scale our systems.

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thank you!

  • @nero4581
    @nero458111 ай бұрын

    Great Video, as always, Anton!

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thanks, if you think anything can be improved, please let me know!

  • @helciopandelo
    @helciopandelo11 ай бұрын

    Truly awesome and simple to learn!!! Thank you!!!

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thank you for the feedback!

  • @dmitriydiachenko5648
    @dmitriydiachenko564811 ай бұрын

    Awesome video! thanks for explaining it

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thanks!

  • @charlesopuoro5295
    @charlesopuoro52957 ай бұрын

    Thank you so much for this clear, insightful explanation of Database Sharding.

  • @AntonPutra

    @AntonPutra

    7 ай бұрын

    Thank you, Charles!

  • @ab_azmi5584
    @ab_azmi55847 ай бұрын

    Great video. Clear & easy to understand.

  • @AntonPutra

    @AntonPutra

    7 ай бұрын

    Thanks Ab!

  • @alonsoalcantar7847
    @alonsoalcantar78472 ай бұрын

    Ive tried to understand sharding for crypto purposes but every "crypto sharding" video display vague descriptions. I stumbled upon this by accident and it was great. Amazing work 🙌

  • @AntonPutra

    @AntonPutra

    2 ай бұрын

    thanks!

  • @bunnybal
    @bunnybal24 күн бұрын

    Really very well explained, thank you very much.

  • @AntonPutra

    @AntonPutra

    24 күн бұрын

    thanks!

  • @shanchessmetilda5550
    @shanchessmetilda55502 ай бұрын

    Great Explanation....Thanks for the efforts

  • @AntonPutra

    @AntonPutra

    2 ай бұрын

    Thanks!

  • @bossgd100
    @bossgd10011 ай бұрын

    thank you for these explanations

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    thanks for visiting

  • @cptn-nemo
    @cptn-nemo3 ай бұрын

    Good explanation, Thanks

  • @meron6913
    @meron691311 ай бұрын

    Great video Anton.

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thank you!

  • @egitim10
    @egitim103 ай бұрын

    Short,nice,clear

  • @LinuxForLife
    @LinuxForLife11 ай бұрын

    Very interesting! Thanks! 👍

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    welcome! my pleasure

  • @erkanakgul6954
    @erkanakgul69545 ай бұрын

    Great explaination! Thanks

  • @AntonPutra

    @AntonPutra

    5 ай бұрын

    thank you!

  • @yourname-scorpion
    @yourname-scorpionАй бұрын

    Great video, thank you

  • @AntonPutra

    @AntonPutra

    Ай бұрын

    thanks!

  • @dhruvpatel6604
    @dhruvpatel660410 ай бұрын

    Excellent tutorial

  • @AntonPutra

    @AntonPutra

    10 ай бұрын

    Thank you!

  • @aslan1504
    @aslan150410 ай бұрын

    As for the hashing, you probably will take hash only of a subset of columns of a record, most probably - only primary key, because changing any field of any record will result the hash to change also, which leads to data losses.

  • @tonnytrumpet734

    @tonnytrumpet734

    4 күн бұрын

    Thanks for this comment. Could you maybe clarify ? By data loss you mean the reverse proxy or whatever way of communicating with databases wont be able to know where to search for the information right ? however the information would still be here. Although it would be extremely computationally hard there would still be way to recover it right ? You could for example recalculate hashes for all the data and redistribute those that aren't belonging to the right database based on the sharding prefix ?

  • @aslan1504

    @aslan1504

    4 күн бұрын

    @@tonnytrumpet734 oh yes, data will still be there, but it basically will become unusable. It's like creating yourself problems to solve.

  • @rayaalkhateeb874
    @rayaalkhateeb8747 ай бұрын

    thanks a lot for explaining

  • @AntonPutra

    @AntonPutra

    7 ай бұрын

    my pleasure!

  • @nicgeorge6126
    @nicgeorge612611 ай бұрын

    Fortunately I’ve been able to get by with two read databases and a write by using table partitioning up until this point. Hopefully I don’t have to tackle sharding any time soon! Great video and thanks for sharing

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thanks! There a lot of distributed databases based on postgres that can shard for you.

  • @vikasgoel7529
    @vikasgoel75295 ай бұрын

    Excellent described

  • @AntonPutra

    @AntonPutra

    5 ай бұрын

    Thanks!

  • @EpoxyArt
    @EpoxyArt4 ай бұрын

    Hi, Anton! How can I search by the field that is not shard key? I need to go thru all the shards? And what if I need to scale it up or down (change shards number)?

  • @tejapolisettysai6950
    @tejapolisettysai69508 ай бұрын

    sir thanks for the video, what do you use for editing, its really good.

  • @AntonPutra

    @AntonPutra

    8 ай бұрын

    thanks, adobe suite

  • @69k_gold
    @69k_goldАй бұрын

    Let's say I'm using a shard-nothing architecture, now let's say there's a relationship between customers table, payments table and orders table. Customers and orders tables are linked by the foreign key customers->id ~ orders->customer_id Orders table and payments table has the foreign key orders->id ~ payments->order_id Now how would you shard this database? You can't use a single shard key, because both customer_id and order_id are important that ensure all the related rows are in a single shard. So how would you solve this problem?

  • @ProSunnySharma
    @ProSunnySharma8 ай бұрын

    Excellent! What tool do you use to do animations?

  • @AntonPutra

    @AntonPutra

    8 ай бұрын

    Thanks Adobe suite

  • @ricardorqr
    @ricardorqr11 ай бұрын

    How do you create the animation for your videos? They are so cool!!!!! 💪🏼

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    Thanks! I use adobe suite.

  • @MagDag_
    @MagDag_5 күн бұрын

    Классное видео, спасибо. Какая стоимость ваших услуг? Нужна консультация для Homelab.

  • @AntonPutra

    @AntonPutra

    4 күн бұрын

    Privet, spasibo! At this point, I can offer one-on-one sessions. I ask that you send me any questions before the session so that I can prepare some examples, and we can go through them during the meeting. I charge $100 per 1 hour session. If you are interested, pls send me an email.

  • @user-ui5yw6pf5p
    @user-ui5yw6pf5p3 ай бұрын

    you said sharding have unique data sets if one sharding not respond then other sharding response you but if customer search record and that record will be in sharding 1 . After that sharding 1 will not respond then what we have to show for customer

  • @user-yv6ti2wf7c
    @user-yv6ti2wf7c5 ай бұрын

    thanks, but i have a question if i use range-based sharding and conside 4 shard what happen if i want to convert to 40 shard? what happen for previous data, and new data [ first i have 3 shard 1(a-h) 2(i-p) 3(q,z)) now need to make it 40.

  • @AntonPutra

    @AntonPutra

    5 ай бұрын

    If you shard manually at the application level, you need to write logic to rebalance it yourself. It's easier to use built-in mechanisms for sharding.

  • @user-yv6ti2wf7c

    @user-yv6ti2wf7c

    5 ай бұрын

    @@AntonPutra thanks

  • @gcheese25
    @gcheese258 ай бұрын

    great video! start subscribing now

  • @AntonPutra

    @AntonPutra

    8 ай бұрын

    Thank you!

  • @rahuldinesh2840
    @rahuldinesh28409 ай бұрын

    Can I do sharding in WordPress database?

  • @AntonPutra

    @AntonPutra

    9 ай бұрын

    WordPress uses a MySQL database under the hood. Take a look at Vitess.

  • @noahgsolomon
    @noahgsolomon9 ай бұрын

    w video

  • @AntonPutra

    @AntonPutra

    9 ай бұрын

    thanks

  • @andherium
    @andherium11 ай бұрын

    I think 99.9% use case are served fine by a monolith database server. Heck even stack overflow is fully powered on a single server

  • @cariyaputta

    @cariyaputta

    11 ай бұрын

    Yes. Still, it's useful to have a knowledge of whatever jargons uppermamagements are throwing at you.

  • @AntonPutra

    @AntonPutra

    11 ай бұрын

    For personal projects, sure, but in the enterprise, you frequently have to deal with sharding.

  • @thewaygoesup
    @thewaygoesup10 ай бұрын

    your example confusing between database shard and table partitioning range-base sharding is about one table sharding not about database sharding.

  • @AntonPutra

    @AntonPutra

    10 ай бұрын

    Noted, will improve

  • @APoIIy
    @APoIIy9 ай бұрын

    So, sharding is a pain in the ass and requires a lot of configuration, analytics and also business logic to manage shards in an respectable way. This is also why NoSQL Databases come in handy as they can scale better horizontaly without this extensive configuration activities you have with traditional SQL databases. But to be said SQL Databases will probably cover 90% of all usecases anyway without you getting into sharding.

  • @AntonPutra

    @AntonPutra

    9 ай бұрын

    Nowadays, once a year, I see a new distributed database based on PostgreSQL come out, lol.

  • @yoggg932
    @yoggg9328 ай бұрын

    Feels like a bot is reading the script. Good content, but please act it out a bit.

  • @AntonPutra

    @AntonPutra

    8 ай бұрын

    thanks for the feedback

Келесі