What is Database Sharding?
Ғылым және технология
🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
► LinkedIn: / anton-putra
► Twitter/X: / antonvputra
► GitHub: github.com/antonputra
► Email: me@antonputra.com
▬▬▬▬▬▬ Related videos 👨🏫 ▬▬▬▬▬▬
👉 [Playlist] Kubernetes Tutorials: • Kubernetes Tutorials
👉 [Playlist] Terraform Tutorials: • Terraform Tutorials fo...
👉 [Playlist] Network Tutorials: • Network Tutorials
👉 [Playlist] Apache Kafka Tutorials: • Apache Kafka Tutorials
👉 [Playlist] Performance Benchmarks: • Performance Benchmarks
👉 [Playlist] Database Tutorials: • Database Tutorials
▬▬▬▬▬▬▬ Timestamps ⏰ ▬▬▬▬▬▬▬
0:00 What is database sharding?
0:12 Why is database sharding important?
0:58 What are the benefits of database sharding?
1:57 How does database sharding work?
3:42 What are the methods of database sharding?
3:47 Range-based sharding
4:59 Hashed sharding
5:52 Directory sharding
6:38 Geo sharding
7:25 How to optimize database sharding for even data distribution?
7:45 Cardinality
8:05 Frequency
8:24 Monotonic change
▬▬▬▬▬▬▬ Source Code 📚 ▬▬▬▬▬▬▬
► GitHub: github.com/antonputra/tutorials
#kubernetes #devops #cloud
Пікірлер: 80
🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
Hard to believe this top-notch content has very less views. Thanks a lot, Anton!!
@AntonPutra
4 ай бұрын
❤️
@jackfrost8969
3 ай бұрын
probably coz it basically has no volumn
Thank you sir for detail explaination of database sharding. We hope a practical handson of Database sharding will publish soon.
Very helpful. Very confused and to the point! I hope your colleagues who do technical videos would follow your framework. 👍🏾👍🏾👍🏾
Superb explanation, and never strayed off topic.
Nice, exhaustive and short video considering it covers a lot.
No Nonsense, direct to point, covering all cases. Well-compiled video!
@AntonPutra
Ай бұрын
thank you!
Great video! Always an important topic when we think about scale our systems.
@AntonPutra
11 ай бұрын
Thank you!
Great Video, as always, Anton!
@AntonPutra
11 ай бұрын
Thanks, if you think anything can be improved, please let me know!
Truly awesome and simple to learn!!! Thank you!!!
@AntonPutra
11 ай бұрын
Thank you for the feedback!
Awesome video! thanks for explaining it
@AntonPutra
11 ай бұрын
Thanks!
Thank you so much for this clear, insightful explanation of Database Sharding.
@AntonPutra
7 ай бұрын
Thank you, Charles!
Great video. Clear & easy to understand.
@AntonPutra
7 ай бұрын
Thanks Ab!
Ive tried to understand sharding for crypto purposes but every "crypto sharding" video display vague descriptions. I stumbled upon this by accident and it was great. Amazing work 🙌
@AntonPutra
2 ай бұрын
thanks!
Really very well explained, thank you very much.
@AntonPutra
24 күн бұрын
thanks!
Great Explanation....Thanks for the efforts
@AntonPutra
2 ай бұрын
Thanks!
thank you for these explanations
@AntonPutra
11 ай бұрын
thanks for visiting
Good explanation, Thanks
Great video Anton.
@AntonPutra
11 ай бұрын
Thank you!
Short,nice,clear
Very interesting! Thanks! 👍
@AntonPutra
11 ай бұрын
welcome! my pleasure
Great explaination! Thanks
@AntonPutra
5 ай бұрын
thank you!
Great video, thank you
@AntonPutra
Ай бұрын
thanks!
Excellent tutorial
@AntonPutra
10 ай бұрын
Thank you!
As for the hashing, you probably will take hash only of a subset of columns of a record, most probably - only primary key, because changing any field of any record will result the hash to change also, which leads to data losses.
@tonnytrumpet734
4 күн бұрын
Thanks for this comment. Could you maybe clarify ? By data loss you mean the reverse proxy or whatever way of communicating with databases wont be able to know where to search for the information right ? however the information would still be here. Although it would be extremely computationally hard there would still be way to recover it right ? You could for example recalculate hashes for all the data and redistribute those that aren't belonging to the right database based on the sharding prefix ?
@aslan1504
4 күн бұрын
@@tonnytrumpet734 oh yes, data will still be there, but it basically will become unusable. It's like creating yourself problems to solve.
thanks a lot for explaining
@AntonPutra
7 ай бұрын
my pleasure!
Fortunately I’ve been able to get by with two read databases and a write by using table partitioning up until this point. Hopefully I don’t have to tackle sharding any time soon! Great video and thanks for sharing
@AntonPutra
11 ай бұрын
Thanks! There a lot of distributed databases based on postgres that can shard for you.
Excellent described
@AntonPutra
5 ай бұрын
Thanks!
Hi, Anton! How can I search by the field that is not shard key? I need to go thru all the shards? And what if I need to scale it up or down (change shards number)?
sir thanks for the video, what do you use for editing, its really good.
@AntonPutra
8 ай бұрын
thanks, adobe suite
Let's say I'm using a shard-nothing architecture, now let's say there's a relationship between customers table, payments table and orders table. Customers and orders tables are linked by the foreign key customers->id ~ orders->customer_id Orders table and payments table has the foreign key orders->id ~ payments->order_id Now how would you shard this database? You can't use a single shard key, because both customer_id and order_id are important that ensure all the related rows are in a single shard. So how would you solve this problem?
Excellent! What tool do you use to do animations?
@AntonPutra
8 ай бұрын
Thanks Adobe suite
How do you create the animation for your videos? They are so cool!!!!! 💪🏼
@AntonPutra
11 ай бұрын
Thanks! I use adobe suite.
Классное видео, спасибо. Какая стоимость ваших услуг? Нужна консультация для Homelab.
@AntonPutra
4 күн бұрын
Privet, spasibo! At this point, I can offer one-on-one sessions. I ask that you send me any questions before the session so that I can prepare some examples, and we can go through them during the meeting. I charge $100 per 1 hour session. If you are interested, pls send me an email.
you said sharding have unique data sets if one sharding not respond then other sharding response you but if customer search record and that record will be in sharding 1 . After that sharding 1 will not respond then what we have to show for customer
thanks, but i have a question if i use range-based sharding and conside 4 shard what happen if i want to convert to 40 shard? what happen for previous data, and new data [ first i have 3 shard 1(a-h) 2(i-p) 3(q,z)) now need to make it 40.
@AntonPutra
5 ай бұрын
If you shard manually at the application level, you need to write logic to rebalance it yourself. It's easier to use built-in mechanisms for sharding.
@user-yv6ti2wf7c
5 ай бұрын
@@AntonPutra thanks
great video! start subscribing now
@AntonPutra
8 ай бұрын
Thank you!
Can I do sharding in WordPress database?
@AntonPutra
9 ай бұрын
WordPress uses a MySQL database under the hood. Take a look at Vitess.
w video
@AntonPutra
9 ай бұрын
thanks
I think 99.9% use case are served fine by a monolith database server. Heck even stack overflow is fully powered on a single server
@cariyaputta
11 ай бұрын
Yes. Still, it's useful to have a knowledge of whatever jargons uppermamagements are throwing at you.
@AntonPutra
11 ай бұрын
For personal projects, sure, but in the enterprise, you frequently have to deal with sharding.
your example confusing between database shard and table partitioning range-base sharding is about one table sharding not about database sharding.
@AntonPutra
10 ай бұрын
Noted, will improve
So, sharding is a pain in the ass and requires a lot of configuration, analytics and also business logic to manage shards in an respectable way. This is also why NoSQL Databases come in handy as they can scale better horizontaly without this extensive configuration activities you have with traditional SQL databases. But to be said SQL Databases will probably cover 90% of all usecases anyway without you getting into sharding.
@AntonPutra
9 ай бұрын
Nowadays, once a year, I see a new distributed database based on PostgreSQL come out, lol.
Feels like a bot is reading the script. Good content, but please act it out a bit.
@AntonPutra
8 ай бұрын
thanks for the feedback