The Most Complicated Algorithm I've Ever Written: SQLite B-Tree Balancing

Ғылым және технология

In this video we visualize how the SQLite 2.X.X B-Tree balancing algorithm works. Technically it's not exactly the algorithm from SQLite but my own version which I implemented for MKDB. My version is almost a 1:1 Rust copy of the original algorithm implemented in C which you can find linked below.
🌐 LINKS
Algorithm Implementation:
github.com/ant...
Original SQLite Implementation:
github.com/ant...
✉️ CONTACT INFO
Business Email: business@antoniosarosi.io
Contact Email: sarosiantonio@gmail.com
Twitter: / antoniosarosi
Instagram: / antoniosarosi
LinkedIn: / antoniosarosi
🎵 MUSIC
• 【Chillstep】Wayr - Agai...
• Swell
• Could We Say Goodbye
📖 CHAPTERS
00:00 Introduction
00:41 Databases & B-Trees
03:30 The Balancing Algorithm
09:24 Deleting & Balancing
12:03 Final Thoughts
🏷️ HASHTAGS
#programming
#computerscience
#algorithm

Пікірлер: 232

  • @rusyaidimusa2309
    @rusyaidimusa2309Ай бұрын

    There are thousands of beginner videos out there, and very few which really digs into the details. This is fantastic.

  • @robertgaina12
    @robertgaina12Ай бұрын

    Me after writing my first linked list

  • @tony_saro

    @tony_saro

    Ай бұрын

    Lol

  • @isuckatthisgame

    @isuckatthisgame

    Ай бұрын

    Me after forgetting how to write a linked list and then re-learning it and feeling like a God again. 😅

  • @minimumt3n204

    @minimumt3n204

    Ай бұрын

    I remember putting that on my resume when I first learned linked lists 😂😂

  • @tony_saro

    @tony_saro

    Ай бұрын

    ​@@minimumt3n204 Google must've been fighting Amazon and Microsoft to hire you 💀

  • @Nico-lj6qs

    @Nico-lj6qs

    29 күн бұрын

    Me after writting my first binary search.

  • @jholloway77
    @jholloway77Ай бұрын

    I love this channel! Solid engineering that isn't just the basics, nor so high level that anything useful is abstracted away. Instead real algorithms and explanations on their utilization in a real world manner. Keep up the good work

  • @tony_saro

    @tony_saro

    Ай бұрын

    I try to keep it in the middle, too high level would be "B-Trees can self balance, that's it" and too many details would be getting into how the code is actually implementing the ideas that I explain, in this case for example the distribution computation is kinda weird with the way the loops work 😂, but whoever wants to understand the details can read through the source code.

  • @LocalGhost_8080
    @LocalGhost_8080Ай бұрын

    Freaking crazy. All this is super interesting! I like how you explain and guide us through the process. Starting from scracth knowing nothing, but researching and building up until you figure out what to do next

  • @AqoCyrale
    @AqoCyraleАй бұрын

    excellent explanation of b-tree insertion, deletion and balancing, I love your visualizations too, thanks a lot!

  • @cipherxen2
    @cipherxen2Ай бұрын

    I have implemented it in my third year engineering almost 24 years ago. It felt like a great achievement.

  • @user-kc1kz6yu6o
    @user-kc1kz6yu6oАй бұрын

    It is amazing that there are still people interested in algothims since nowdays people are mostly interested in chat gpt and the other AI models. I really appreciate it 🙏 Thank you!

  • @Maric18

    @Maric18

    Ай бұрын

    how are people still interested in chat gpt? most developers i know have turned off the ai assist stuff again as it is need but also kinda distacting i feel like by now its 90% businessy people trying to sell each other AI hype LLMs are cool and all but they aren't world changing yet, we gotta wait a few more months for that

  • @energy-tunes
    @energy-tunesАй бұрын

    Please keep these vids up there are 0 DBMS dev vids

  • @tony_saro

    @tony_saro

    Ай бұрын

    This is gonna be the last database video for now, I'll move to a different project for the next video. All the source code I wrote is available on GitHub and I put a lot of emphasis on documentation for people interested in writing databases, so feel free to read through it if you want to get into the details.

  • @AmineAllab

    @AmineAllab

    Ай бұрын

    @@tony_saro thank you , you are such an amazing developer wish you all luck

  • @axelneumann8443

    @axelneumann8443

    Ай бұрын

    There is a KZread channel called "CMU Database Group"

  • @energy-tunes

    @energy-tunes

    Ай бұрын

    @@tony_saro there are lots of components in your dbms implementation though, i see no reason why you shouldn't make a video on them as well, for example sql parsing

  • @tony_saro

    @tony_saro

    Ай бұрын

    @@energy-tunes I'm reserving parsing for the "Writing my own Compiler" video 😂, it's gonna be much more important there.

  • @miguelrosario2053
    @miguelrosario2053Ай бұрын

    Phenomenal video… very few people make content about the algorithms and technology we take for granted… the db implementation video was insanely great 😊

  • @luisfernanadoperezalvarado
    @luisfernanadoperezalvaradoАй бұрын

    aquí apoyando esta nueva iniciativa

  • @CyberWolf755
    @CyberWolf755Ай бұрын

    You've explained the principles of a B-Tree really well that I would be comfortable trying to make it myself. I think a basic implementation would not be that hard, but a performant one would be substantially harder.

  • @tony_saro

    @tony_saro

    Ай бұрын

    A basic implementation is simpler, you don't need the distribution algorithm.

  • @asjvchnvh9313
    @asjvchnvh931327 күн бұрын

    Thank you for this series of videos, I am starting to build my database too, and by research found your videos. Was really useful!

  • @harleyspeedthrust4013
    @harleyspeedthrust4013Ай бұрын

    i did this shit last year, i had found this book on data structures at work so i had the bright idea to implement them in my favorite language, c++. (i do nodejs at work and i would rather do low level c or c++ so i better get cracked at it before i try switching jobs.) i can safely say that deleting a node from a btree and rebalancing the tree was some difficult shit. it was hella fun though, and I built a lot of cool data structures that i might have a direct use for in 50 years

  • @Rapha_Carpio
    @Rapha_CarpioАй бұрын

    im taking the CS50 SQL and watching this videos is like, me watching what i would be able to understand in a few months, but is 5am and i cant sleep so lets watch this now

  • @cyakimov
    @cyakimovАй бұрын

    Great content Tony. You know you've mastered something when you can ELI5 a concept like this. Keep it up!

  • @xipuulze2391
    @xipuulze2391Ай бұрын

    Nunca comento nada, pero vaya. Todos tus videos son top!. Te sigo desde el otro canal y siempre que puedo le doy un vistazo cuando tengo burnout. Me recuerda que todo es con paciencia y muchos prints jajaja. Felicitaciones, Sarosi. De verdad, increíble!

  • @tony_saro

    @tony_saro

    Ай бұрын

    Una barbaridad de prints 😂

  • @LLGilad
    @LLGiladАй бұрын

    Keep going, you are the best doing the content I saw in KZread for a long time!

  • @okseaj
    @okseajАй бұрын

    This is a really good exercise. I wrote one a few years ago when I was learning about databases myself. I never thought about looking at SQLite source code though, I was implementing it based on the wikipedia article - definitely a grind. Nice to see more resources being created for it.

  • @tony_saro

    @tony_saro

    Ай бұрын

    SQLite is pretty simple compared to MySQL or Postgres so it's great for learning. I'm talking about SQLite 2 though (written in the early 2000s), SQLite 3 is more complex but still simpler than MySQL and Postgres.

  • @smajdovamanka
    @smajdovamankaАй бұрын

    We must protect this man at all costs

  • @tony_saro

    @tony_saro

    Ай бұрын

    🫡

  • @carlos32195
    @carlos32195Ай бұрын

    I remember failing the assignment of doing a b-tree algorithm.

  • @m3mbrillo_
    @m3mbrillo_Ай бұрын

    it's a monton. Excelente video, lo tengo que ver como 2-3 veces para terminarlo de entender.

  • @avi12
    @avi12Ай бұрын

    As a front end dev, this video glued me to the screen

  • @JTB_H
    @JTB_HАй бұрын

    hermano te felicito, que buen vídeo es este, me encanta la informática a bajo nivel y tu contenido apunta a eso mismo, algoritmia, arquitectura de computadores, SO, etc, en lenguajes de programacion de hombres rust c/c++, al área y conocimientos mas puro y duro de las ciencia de la informática, esto si que es contenido de Ingeniero, seguí así.

  • @fblua
    @fblua26 күн бұрын

    Great video Tony. !! Thank you for sharing your knowledge; thank you for your excellent job. !! My best wishes for your channel. Fernando, Buenos Aires, Argentina

  • @qbqbqdbq
    @qbqbqdbq22 күн бұрын

    The time honored tradition of reimplementing perfectly fine software in rust, but worse. Thanks dude.

  • @tony_saro

    @tony_saro

    22 күн бұрын

    Gotta rewrite everything in Rust and make it blazingly slow 🤙

  • @paxdriver
    @paxdriverАй бұрын

    "bored"?!?! This was amazing!! New sub, thank you.

  • @WhoEls
    @WhoElsАй бұрын

    During Uni I found classic Database algorithms always simple compared to the nightmarish stuff needed in Distributed systems.

  • @tony_saro

    @tony_saro

    Ай бұрын

    Any particular algorithm from distributed systems you find nightmarish?

  • @mohamadkawas2287

    @mohamadkawas2287

    Ай бұрын

    Mine was the Guassian Blurr for image processing. We had to do it in C and cuda. I still have nightmares about it.

  • @CaptTerrific
    @CaptTerrificАй бұрын

    Aww this takes me back to my data structures course 20 years ago

  • @ClaytonTownley
    @ClaytonTownleyАй бұрын

    I once spent an hour solid going thru the SQLite source code, also not understanding what I was looking at. Still don't. But it gives you an appreciation for the genius of D. Richard Hipp (creator of SQLite). Anyway, this is a cool project and I'm motivated to make my own simple database. Thanks!

  • @eldiegoasecas
    @eldiegoasecasАй бұрын

    this is insane, really impressive

  • @soumojitchakraborty5021
    @soumojitchakraborty5021Ай бұрын

    Also please keep sharing your experience about developing mkdb. your first video was very informative🙂🙂

  • @mlab3051
    @mlab3051Ай бұрын

    We need more of video like this!! Good stuff.

  • @aron-gx9mh
    @aron-gx9mhАй бұрын

    this is really impressive. it's almost like software engineers are still needed and ai isn't going to take over any time soon. lol

  • @tony_saro

    @tony_saro

    Ай бұрын

    AI won't correctly implement an algorithm like this one from scratch any time soon 😂

  • @aron-gx9mh

    @aron-gx9mh

    Ай бұрын

    agreed. i still have work to do. the only thing i need to watch out in the ai world is which ai is giving me less errors.

  • @Alexander-zt9kz

    @Alexander-zt9kz

    Ай бұрын

    This isnt really even software engineering either. Very few software engineers write these type of stuff, as fascinating as it is.

  • @tony_saro

    @tony_saro

    Ай бұрын

    @@Alexander-zt9kz I'd say it's actually more "engineering" than application development, at least in the traditional meaning of the word "engineering". You have the theory, or the "science" which tells you this data structure has such and such time complexity / space complexity, the data structures must comply to such and such rules, there's some random paper that proves it, and now good luck translating that to code that actually runs on the CPU. Mechanical or electrical engineers are supposed to apply math and physics to real world problems, so shouldn't "software" engineers apply math and computer science to real world problems? The reality though is that most of us build APIs and we're sending JSONs back and forth 😂😂😂.

  • @hendrickginn3323

    @hendrickginn3323

    Ай бұрын

    Ah …not so sure about that. AI has started to develop algorithms…Ai has already created one to improve matrix multiplication. Something we thought was already optimized. Don’t deny our soon to be lord and savior.

  • @dearozero3631
    @dearozero3631Ай бұрын

    Beautiful narration for the algorithm.

  • @Nico-lj6qs
    @Nico-lj6qs29 күн бұрын

    Tus videos son buenísimos como explicas y las animaciones que utilizas hace que sea fácil de entender, yo estoy haciendo una app web con java para visualizar las operaciones en arboles B que indique todas las lecturas/escrituras. Complete la inserción y hacer la realocación de punteros cuando un nodo se divide y promociona a medida que el árbol crece fue bastante mas complicado de lo que pensaba, pero ni de cerca es tan recursivo y elegante como el que mostras acá 😅

  • @MrMeltdown
    @MrMeltdownАй бұрын

    20 years ago I studied data structures and algorithms. I remember red black trees and decided I would try and do a b-tree for laughs… I lasted a day before giving up and going down the pub. We’ll done on your success. I’m pretty convinced I made the right decision. The enlightening part was the special case of the toot node. I’m pretty sure that is where I was getting confused. I might even try my own implementation… though I may end up on anti -depressants this time round…

  • @tony_saro

    @tony_saro

    Ай бұрын

    20 years ago you couldn't use the internet to find implementations or explanations about how the algorithm works, at least not as many as today, all you had was probably books. I don't think I'd be able to write a database in 2004 😂.

  • @christopherprobst-ranly6357
    @christopherprobst-ranly6357Ай бұрын

    B trees are CS 101 structures. They are taught in 2/3 semester BA. Go into Media Codecs / Compression or Cryptography, that‘s the definition of complex. But you explained it very good and understandable, thats the art of teaching and this helps understanding.

  • @AG-ur1lj

    @AG-ur1lj

    Ай бұрын

    That makes them 102/103 structures. Linked lists and heaps are 101

  • @tony_saro

    @tony_saro

    Ай бұрын

    Depends on what you understand by B-Tree. A simple in-memory implementation of a B-Tree (not a variant like B* Tree) might be closer to CS 101. A B* Tree implemented on disk with a caching system that loads pages from disk into memory and evicts dirty pages is not "CS 101". Judge by yourself, here's the implementation, if this looks like CS 101 to you then you must be some sort of genius😂 github.com/antoniosarosi/mkdb/blob/master/src/storage/btree.rs

  • @AG-ur1lj

    @AG-ur1lj

    Ай бұрын

    @@tony_saro It does not depend on what any individual means by ‘B-Tree,’ nor does it depend on that commenters intellect. MIT is one of the very top engineering schools on the planet, and you can see it right there in the syllabus. 6.046 - _Design and Analysis of Algorithms_ - Week 2 in syllabus: B-Trees - Prerequisites: 6.006 - _Intro to Algorithms_ It’s 102 everywhere. Saying it’s 101 looks stupid to anyone that’s already taken those courses.

  • @tony_saro

    @tony_saro

    Ай бұрын

    @@AG-ur1lj Okay but that's just an algorithms course, I had one similar at the uni I went to that involved data structures other than B-Trees but more complicated than 101 data structures. And as I said, that course will probably have an assignment that requires students to implement an in-memory B-Tree, most likely with fixed size data. Once you get into variable length data and disk storage, complexity spikes. Considering something 101 obviously depends on the exact implementation, a basic hash map is tought in most 101 courses but a concurrent disk hash map used by databases for hash indexes is not "101". As I said, judge by yourselves though, compare your 101 or 102 assignments to the code I wrote and you'll see the difference.

  • @AG-ur1lj

    @AG-ur1lj

    Ай бұрын

    @@tony_saro I’m on “your side,” I’m just not willing to grant any benefit of the doubt to OP-even if you were only doing so to be humble. I think his definition of “complexity” is bad. I think his examples of complexity suggest that it’s unlikely he’s ever implemented a B-Tree for any real use case. I also think he made those comparisons to imply that the content of this video is easy-just kid’s stuff. I don’t just disagree with the original post, but tbh it pisses me off a little. Not only is he objectively wrong about the level of the material-which again, is basically halfway through an undergrad program and well beyond what any of the fresh webdev Bootcamp applicants have seen-he’s also being condescending towards people with the drive and discipline to self-learn while also missing the entire point. Defining a concept in a _perfectly_ simple way is exactly what makes any algorithm clever. That’s what allows B-Trees to exist. That _is_ how you find the broader, more generalized concept. TL;DR: 102 isn’t an insult. I would have to spend a full year of my income before even being eligible to sign up for Algos 102 at any college. Edit: thanks for the video, by the way. Great upload. I probably would’ve stayed out of everything if I’d known you were out responding to the h8ers.

  • @hoxuro
    @hoxuroАй бұрын

    Buen video Antonio, ahora puedo aprender backend e inglés a la vez. Éxitos.

  • @saravanasai2391
    @saravanasai2391Ай бұрын

    Wow, That is a great explanation. B - trees are advanced version of 2-3 trees. Which holds most of the properties of b-tree. I have written AVL trees on my own. So, To my understanding up and till this point, a b-tree or even a binary can be represented using a linear data structure like an array using an expression of (2*i)- left child & (2*i)+1- right child. In contrast, (i) is an index of the current element in the array. So, can leverage these to store information on disk as we know in storage we can get access to a 4kb page for each read operation and 4kb is size assured by hard disk manufacturers that if the power goes off. It can write 4kb data to disk without losing it. so, it helps us with crash recovery. I am trying to build a very simple database without complex data structures to make clear how this disk stores data & is retrieved at a low level. I am from a PHP background. I know some basics of c programming & javascript. Why do you choose Rust to build this database? why not c or C++.

  • @tony_saro

    @tony_saro

    Ай бұрын

    C doesn't have a standard library so you have to write a lot of code or use third party libraries. The standard library of Rust allowed me to write the entire database with 0 dependencies. On the other hand it has modern features that C lacks (genetics, pattern matching, closures, traits). C++ I guess you can use it, but I don't know any C++ I've never used it. I chose a language I already knew.

  • @winfle

    @winfle

    Ай бұрын

    Because C is outdated feature languge

  • @schabby30
    @schabby30Ай бұрын

    Hi, you and your channel are amazing! Thanks for this content really. One question, if I may. Could/would you consider doing some more 'in depth' coding walk throughs we could even follow along with you? I'd personally love to see more code, even if it means longer videos and/or topics less complex than b-tree balancing 😁 But even this code I'd really like to see more of. It's a wild one man. Amazing work!

  • @tony_saro

    @tony_saro

    Ай бұрын

    I am not sure how to do that, I wouldn't do 1 hour tutorials of basic "hello world" projects, there are a lot of channels that already do that. Streaming is another option, maybe I can stream smaller projects that would take max 10 hours / streams and then reupload to KZread. But for now I'm focused on explaining ideas/concepts, trust me, as I said at 12:11, if all you have is the code then you'll have a very hard time figuring out how it works. I don't even know how many hours I spent understanding this particular algorithm but in total I spent months with it. If I had had a simple 12 minute video like this one I would've probably spent days instead of months. When you write code you also have to get into the details of the code, not only do you have to explain the idea behind the code but also how each line works. For this particular algorithm, explaining the code would probably take 12 hours instead of 12 minutes, because you also have to understand other parts of the code base such as the pager and cache system. The code base is 25K lines, code only is probably 12-15K, the rest is documentation. So projects like this one would be impossible to stream / teach, it would take hundreds of hours to write and explain all the code. Simpler projects might work, as long as I'm not getting into "hello world" territory. In the meantime, if you want to read the code, it's on GitHub, it's thoroughly documented, some parts of the code are explained in my videos, so it should be easier to follow than most other codebases.

  • @ketanchaudhari2337
    @ketanchaudhari2337Ай бұрын

    Amazing 🤩, Really appreciate the hard work.

  • @abdelwakyl-benterki
    @abdelwakyl-benterkiАй бұрын

    "i don't think anything will come close to that in the near future" *B+ Trees sipping a beer in the corner...*

  • @tony_saro

    @tony_saro

    Ай бұрын

    I mean from my perspective, I won't code anything as complicated as databases anytime soon. B+ Tree shouldn't be that complicated with all the code I have, the algorithm from this video is reusable, the overall structure would require many changes but going from a B-Tree to a B+ Tree is not as complicated as going from nothing to a B-Tree.

  • @abdelwakyl-benterki

    @abdelwakyl-benterki

    Ай бұрын

    @@tony_saro fair enough, i didn't code neither of the two, i just know how they work, so shout out to you for the effort.

  • @StevenHokins
    @StevenHokinsАй бұрын

    Love this series, thank you ❤

  • @sorcdk2880
    @sorcdk288028 күн бұрын

    Just wait until you realise there are special optimized algorithms for when you are looking up more than one key at the same time. Those will make B-trees look like cute little things in comparison.

  • @tony_saro

    @tony_saro

    28 күн бұрын

    I implemented some ranged queries, for statements like "primary_key BETWEEN X AND Y" or "primary_key > X". Other than that you can't do much else with B-Trees.

  • @sorcdk2880

    @sorcdk2880

    28 күн бұрын

    I think I need to correct that I mean the kind of datastructures you use to optimize those searching. You typically end up with some variation of a k-d tree, which you then need to turn into a page optimized version. The optimized ones also have "fun" optimizations such as fractional cascading, which is somewhat of a nightmare.

  • @tony_saro

    @tony_saro

    28 күн бұрын

    Oh I'm pretty sure there are data structures more complicated than this one. I'm just speaking from experience, I did not implement any of them. I wrote some complicated data structures for a memory allocator project, but not as complicated as disk B-Trees.

  • @sohansingh2022
    @sohansingh2022Ай бұрын

    Beautiful piece of art!

  • @nguyenhuynh3792
    @nguyenhuynh3792Ай бұрын

    I learnt about this and red black tree but have never implemented them. Will implement them soon.

  • @tony_saro

    @tony_saro

    Ай бұрын

    Good luck 👍

  • @user-gh1cq5rl3h
    @user-gh1cq5rl3hАй бұрын

    Awesome video Antonio, Goat 🐐

  • @pezo1919
    @pezo1919Ай бұрын

    Amazing work, please keep it up! :)

  • @a7mdbest15
    @a7mdbest15Ай бұрын

    this channel is a hidden gem

  • @iamSergioCampbell
    @iamSergioCampbellАй бұрын

    I like you trasnparency. Keep doing 👌🏽

  • @rockNbrain
    @rockNbrainАй бұрын

    Great job dude! Please give us more of this senior juice lol

  • @bamberghh1691
    @bamberghh1691Ай бұрын

    Hi, I have several questions about databases in general: Do the b-trees store only the references to the data (and it's stored somewhere else in the file in a heap-like fashion) or are they also used to store the actual data? How are the variable-sized data types like TEXT/BLOB stored? What happens when they're inserted/modified/deleted?

  • @tony_saro

    @tony_saro

    Ай бұрын

    My full database video answers some of your questions. I'll link it below and I'll answer anyway. kzread.info/dash/bejne/Z4SXk5qglZzVmKw.htmlsi=uTQ94tOYGIk4cQuo B-Trees store the entire rows, all the data. Text and blobs are stored using overflow pages, you basically break the data into multiple chunks, store each chunk in a different page outside the B-Tree and link all the pages. Inserting texts or blobs is no different than inserting any other field, you deallocate or reuse the pages previously occupied by the blobs if you're removing or modifying, and when you're inserting you just use the algorithm from the video.

  • @lf6190
    @lf619027 күн бұрын

    "I decided to write my own database" ... lol how much red bull were you drinking that day? XD

  • @tony_saro

    @tony_saro

    27 күн бұрын

    Yeah you don't wanna know

  • @mr.nightstar
    @mr.nightstarАй бұрын

    Fantastic content!

  • @technophylic3321
    @technophylic3321Ай бұрын

    Hey , great work man ! Btw which tool you use for animation?

  • @tony_saro

    @tony_saro

    Ай бұрын

    Adobe Premiere

  • @pauek
    @pauek29 күн бұрын

    Great video! Great animations and visuals (is it manim?)! Great explanation! Great pace and rhythm. But the code font is too close to Comic Sans for me... 😅

  • @tony_saro

    @tony_saro

    29 күн бұрын

    Not Manim, it's just standard Adobe Premiere. Code Font has many haters 😂

  • @CbAqvq191
    @CbAqvq191Ай бұрын

    Hermano, buenísima explicación!!!

  • @JTB_H
    @JTB_HАй бұрын

    Te felicito por haberte expandido al habla inglésa, muy buen contenido .

  • @LucasBatistussi
    @LucasBatistussi25 күн бұрын

    Removal in BTree is a nightmare

  • @tony_saro

    @tony_saro

    25 күн бұрын

    Just a bit hahaha. This is the worst part, took me a lot of time to realize: github.com/antoniosarosi/mkdb/blob/bf1341bc4da70971fc6c340f3a5e9c6bbc55da37/src/storage/btree.rs#L821

  • @miketag4499
    @miketag449920 күн бұрын

    I have two questions: 1. In case of underflow root, ehy the page numbers are not modified when the depth gets decreased by 1? In that case the child pages have a wrong page number 2. Are the indexes loaded into a btree upon initialization? Thanks

  • @tony_saro

    @tony_saro

    20 күн бұрын

    Only the page pointers are modified when reorganizing entries.

  • @tony_saro

    @tony_saro

    20 күн бұрын

    Indexes are not "loaded" upon initialization, they are created with the "CREATE INDEX" statement and then they exist on disk.

  • @tech-nomade
    @tech-nomade21 күн бұрын

    Dude that's dope! Could you share with us how you build such good animations? Just After Effects / Davinci Fusion? Or any additional tools? How much time do you spend per minute of final video? I guess at least 1-2 hours per minute?

  • @tony_saro

    @tony_saro

    21 күн бұрын

    Yeah it's about 1-2 hours per minute. I don't even use After Effects it's just Adobe Premiere with the "Essential Graphics".

  • @tech-nomade

    @tech-nomade

    21 күн бұрын

    @@tony_saro Hey man, thanx for responding that quick! I have same Wallpaper as you @ 12:09 on the right Display! :D Did you learn this kind of video production with basic Premiere Pro tutorials or are there special kind courses? Can you suggest any to learn from? Or any payed / premium ressources? I want to create vids like yours / let's get rusty's /Dreams of Code's etc on Linux, FOSS, Vue, Golang and Rust as well, but can't find the right resource to get started. Anyhow, subscribed!

  • @tony_saro

    @tony_saro

    21 күн бұрын

    I didn't use any courses at all, you can start with a basic Premiere tutorial on KZread and after that search for specific tutorials, "how to do X in Premiere". That's all I did, but keep in mind these are not my first videos, I have another KZread channel in Spanish which I started in 2019, so I have a lot of experience with editing.

  • @tech-nomade

    @tech-nomade

    21 күн бұрын

    ​@@tony_saro thanx man! yeah, probably just need to get started and then it comes from alone. Going to do a Davinci Resolve course next days on Udemy (canceled my Adobe CC long before I switched to Linux from Mac - even more evil to me than Apple, Google & Co, on the same level as Oracle). Looking forward to more vids on your projects here on KZread!

  • @fernando6547
    @fernando654722 күн бұрын

    Hace poco lo implemente en java y fue bastante complicado,sin la recursión no podría hacerlo La insercion y eliminacion de un Árbol B fue realmente un reto

  • @tony_saro

    @tony_saro

    17 күн бұрын

    La eliminación especialmente es complicada. La inserción también pero tiene menos casos excepcionales.

  • @_CJ_
    @_CJ_Ай бұрын

    I may be weird but I like this algorithm and even if it seems complex I have some urge to implement it myself :D I remember from university when we went through C and we implemented a lot of basic trees and algoritms and it was fun (until some memory problem appeared :D) Nowadays I just use some library from someone a lot smarter who is able to use math to prove it. Thank you for video - again cool animations and very easy to follow. Cheers 👍

  • @tony_saro

    @tony_saro

    Ай бұрын

    You'll learn a lot if you implement it but it requires a lot of effort. You need to implement the entire B-Tree if you want to write this algorithm 😂, and you need a system that loads pages from disk into memory and writes them back, SQLite calls that system the "pager".

  • @_CJ_

    @_CJ_

    Ай бұрын

    @@tony_saro Yea, sounds like I'll pass pager and implement just B-tree part with numbers/strings for fun :D I don't feel that good about low-level IO operations :D I would like to check Rust or Zig when I have time so that could be good combo

  • @s8x.
    @s8x.Ай бұрын

    so many steps and cases

  • @mohannadqa5101
    @mohannadqa510123 күн бұрын

    Quality❤

  • @framegrace1
    @framegrace1Ай бұрын

    Hahaha... i remember doing this B-Tree rotations on the uni. But didn't seemed so dificult back then. Have to try again and see how rust I've become.

  • @tony_saro

    @tony_saro

    Ай бұрын

    The rotation itself is not complicated. The complicated part is generalizing the rotation for variable length data and page allocation/deallocation.

  • @inraid
    @inraid16 күн бұрын

    SQLite is a perfect example how not to write a database.

  • @tony_saro

    @tony_saro

    16 күн бұрын

    According to who?

  • @corsaro0071
    @corsaro0071Ай бұрын

    Thanks for the great content

  • @tony_saro

    @tony_saro

    Ай бұрын

    Thank you for the super

  • @mwave3388
    @mwave3388Ай бұрын

    I like rectangles.

  • @tony_saro

    @tony_saro

    Ай бұрын

    Ah yes, rectangles 🟦

  • @cdavidd
    @cdaviddАй бұрын

    gracias por los subtitulos crack

  • @phv8753
    @phv875328 күн бұрын

    Who knows prof Saraj Sahni behind this Btree algo?

  • @winfle
    @winfleАй бұрын

    Me, when I've passed a variable with a reference

  • @GamerNeko20
    @GamerNeko20Ай бұрын

    por fin señales de vida

  • @tony_saro

    @tony_saro

    Ай бұрын

    🤙

  • @mayfly0
    @mayfly0Ай бұрын

    amazing video 👍👍👍

  • Ай бұрын

    What a great video!

  • @marks95
    @marks9515 күн бұрын

    wow! encontré el canal de antonio sarosi, podriás añadir con la nueva función de youtube, pistas de audio, para que también llegue a mas audiencia.

  • @tony_saro

    @tony_saro

    15 күн бұрын

    No la tengo disponible.

  • @arifansari3005
    @arifansari3005Ай бұрын

    If a page overflows while inserting, how does it fit into the page? Like if my page is just 4KiB how is it possible to allow overflow even temporarily. Ps: I’m trying to make my own dbms from scratch. I had the idea like months ago. Your videos have been a great help, thanks. Keep uploading!

  • @tony_saro

    @tony_saro

    Ай бұрын

    Good question, the exact implementation is a little intricate, but a basic straight forward idea would be, just use a growable array, Vec in Rust. For performance reasons, I used something more elaborate, here's the direct link to the documentation: github.com/antoniosarosi/mkdb/blob/bf1341bc4da70971fc6c340f3a5e9c6bbc55da37/src/storage/page.rs#L1023

  • @pollafattah7062
    @pollafattah7062Ай бұрын

    That is brilliant

  • @kikisxx
    @kikisxxАй бұрын

    Buen vídeo, bro.

  • @diegom4363
    @diegom4363Ай бұрын

    Hey this is amazing! Do you mind sharing the source where you learn from? Be it book, website, course or whatever you used to actually learn to implement this algorithms and the whole database

  • @tony_saro

    @tony_saro

    Ай бұрын

    I explained it in the full database video, check it out. The resources are also linked in the README that's on GitHub. But basically I used the "CMU Intro to Database Systems" course and the code of SQLite.

  • @luscasleo
    @luscasleoАй бұрын

    Amazing content

  • @rasputindasilva858
    @rasputindasilva858Ай бұрын

    This indices corresponde to offsets in a file ? I mean indice 5 for instance corresponde to x bytes from the bigining of the file ?

  • @tony_saro

    @tony_saro

    Ай бұрын

    No, 5 is the primary key of the row. There are no offsets in this video.

  • @brettbyrnes577
    @brettbyrnes577Ай бұрын

    You convinced me to never write my own DB - thank you! 🙏 🙏 🙏

  • @tony_saro

    @tony_saro

    Ай бұрын

    You're welcome 😂

  • @user-yi6sb8qo1j
    @user-yi6sb8qo1jАй бұрын

    Great viz. This is quite different from the B-tree as I understand in book, maybe there are different variants and implementations like B+ tree B* tree. I need to review. Is file system work in similar way? I know I can look it up though

  • @user-yi6sb8qo1j

    @user-yi6sb8qo1j

    Ай бұрын

    Yes, file system works in similar way and explains why there is limited number of C:, D: drive for us to choose in Windows file system formatting. There is also B# tree. I have no idea about it at all. B in Btrees, is possibly for Bayer-McCreight, 1972

  • @user-yi6sb8qo1j

    @user-yi6sb8qo1j

    Ай бұрын

    I think implementing a self balancing tree is medium size project, meanwhile I may choose to solve 75 Leetcode problems

  • @tony_saro

    @tony_saro

    Ай бұрын

    Technically this is a B* Tree, but it's just a name that Knuth came up with in his book "Sorting and Searching".

  • @user-yi6sb8qo1j

    @user-yi6sb8qo1j

    Ай бұрын

    @@tony_saro Good Classic Reference, What I talked about is from Sedgewick's, Sedgewicks is student of Knuth (Ka nooth)

  • @WarchantUA
    @WarchantUAАй бұрын

    Are btrees shallow generally shallow? The algorithm probably has some configs, like the node capacity, nr of children.... how to choose them? In theory, the more capacity the nodes have, the less depth tree will have, is this correct?

  • @tony_saro

    @tony_saro

    Ай бұрын

    It's correct. If by shallow you mean not deep, then yes B-Trees should be shallow. It's impossible to calculate the exact depth since nodes store variable entries so you don't know how many entries each node will hold. But there's a configuration for that, each node must store a minimum of entries, in my code it defaults to 4 entries. So root has a minimum of 5 children, then each child has at least 5 children, so third level is already 25 pages. The depth will grow very slowly. The node capacity is determined by the page size, which is a power of two between 4 KiB and 64 KiB. The greater the page size, the more entries each node will store so less depth.

  • @DevelTime
    @DevelTimeАй бұрын

    Thank you very much for sharing, it is great you continue this topic. Unrelated question, only now I noticed your chair. What brand/type it is? Are you satisfied with it? I am asking because my back is killing me (too many hours in front of monitor) so I am looking something sane 🙂

  • @tony_saro

    @tony_saro

    Ай бұрын

    It's Razer Iskur. It's not a bad chair but it's pretty expensive, if you want a good chair and you have enough money you should probably get an ergonomic chair like the ones made by Herman Miller, that's what I'd buy if I had to get a new chair right now. But don't take my word for it, I haven't personally tried those chairs, I just heard good things / reviews about them. Gaming chairs are mostly for aesthetics.

  • @DevelTime

    @DevelTime

    Ай бұрын

    @@tony_saro Many thanks for your honest reply, it is very helpful. Yes, I also heard heard about HM, one thing that stops me more than price is weight -- this is wrong direction, steel and heavy bicycle is around 13kg, digital piano 14.5kg, a chair something like 20kg 😀

  • @tony_saro

    @tony_saro

    Ай бұрын

    ​@@DevelTimeWell the Razer Iskur is 30Kg 😂, so I wouldn't have a problem with weight, especially if the chair is comfortable.

  • @ashwdq
    @ashwdqАй бұрын

    GOAT🔥🔥🐐🐐

  • @d4rkmn643
    @d4rkmn643Ай бұрын

    on my dsa course at university we learned about btrees, and usually to learn about programming i prefer learning by doing instead of just listening to theory. bad idea, could only manage to implement an insertion on btree after like two days of trying to get it to work, and even then i couldnt manage deletion. then also came the b+ trees and i couldnt do it lmao

  • @tony_saro

    @tony_saro

    Ай бұрын

    I also learn by doing but if you literally have no idea what you need to do it's kinda difficult 😂. This algorithm in particular I just read through SQLite, added my own comments, print statements, modifications, etc, until I understood how it works. Then I made my own version.

  • @user-yi6sb8qo1j

    @user-yi6sb8qo1j

    Ай бұрын

    Deletion on Symbol Table is hard. Refer to Sedgwick’s lecture, LLRBT deletion took years to complete! I tried hard to go through why it works

  • @bossgd100
    @bossgd100Ай бұрын

    Do you have links of learning ressource you used ?

  • @tony_saro

    @tony_saro

    Ай бұрын

    Go to the GitHub repository, they are linked there. The respiratory is in the description of the video.

  • @bossgd100

    @bossgd100

    Ай бұрын

    @@tony_saro cool thank you !

  • @jesushurtado1560
    @jesushurtado156018 күн бұрын

    Ahhhhh, ahora te la tiras de gringo. Jajaja buenísima, gracias por el vídeo

  • @tony_saro

    @tony_saro

    18 күн бұрын

    Jajajaja 🇺🇸🦅

  • @danielgilleland8611
    @danielgilleland8611Ай бұрын

    Bravo!

  • @Mohamed_MDJ
    @Mohamed_MDJАй бұрын

    We studied balanced trees in second year in cs , and it was brain cracking 💀💊, that was because we learned in heavy subject (file structures and data structures), wich i saw useless untill 4 months later

  • @bart2019
    @bart2019Ай бұрын

    Moving all that data around sounds rather inefficient.

  • @tony_saro

    @tony_saro

    Ай бұрын

    If you don't move it fragmentation will cause the height of the tree to increase (worse performance on queries) and you'll use a lot of disk space for nothing (a lot of pages at 50% or below). And it's not really a lot of data if you think about it, pages are 4 KiB by default and the tree should not be that deep considering it grows logarithmically, after you insert something you've basically cached all the pages down to the leaf in memory, you move stuff around in memory only and then run the I/O on disk. It's pretty clever actually.

  • @amj864
    @amj864Ай бұрын

    commenting for the algorithm

  • @tony_saro

    @tony_saro

    Ай бұрын

    Thanks 📈

  • @stingraywww
    @stingraywwwАй бұрын

    Ya pero qué fondo usas?🤔

  • @tony_saro

    @tony_saro

    Ай бұрын

    😂😂😂😂😂 nooo, otra vez no jajaja

  • @SETHthegodofchaos
    @SETHthegodofchaosАй бұрын

    so how did you do the animations? :D

  • @tony_saro

    @tony_saro

    Ай бұрын

    Adobe Premiere

  • @alfredonovoa8124
    @alfredonovoa8124Ай бұрын

    Most database management systems use B+Trees instead of B-Trees because the B+tree holds no data in the internal nodes. This maximizes the number of keys stored in a node minimizing the number of levels. Scans are easier because leaf nodes are linked and deletes are also easier because all the data is in the leaf nodes.

  • @tony_saro

    @tony_saro

    Ай бұрын

    You still have to balance internal nodes with B+Trees. This algorithm is applicable to B+Trees with some modifications. I used normal B-Trees because the implementation is simpler than B+Trees, but then it complicates some other parts of the code. Sequential scans of entire tables are more complicated for example, and less efficient as well.

  • @GabrielxC
    @GabrielxC21 күн бұрын

    Que paso con el español?

  • @tony_saro

    @tony_saro

    20 күн бұрын

    instagram.com/s/aGlnaGxpZ2h0OjE3ODUxMDI2NDY1MjQ5NzA1?story_media_id=3399154947095379898_1492796422&igsh=bzl3MTJ4ZTNvNHd4

  • @energy-tunes
    @energy-tunesАй бұрын

    Isnt this B* tree? B tree doesn't borrow space from siblings when insertion overflows

  • @tony_saro

    @tony_saro

    Ай бұрын

    Yeah, but that's just how Knuth named this variant in his book. Technically it is a B*Tree, but nobody even knows what the B stands for 😂

  • @J0s3p029
    @J0s3p029Ай бұрын

    Llevo un año sin saber que te habías pasado a la comunidad inglesa XD

  • @tony_saro

    @tony_saro

    Ай бұрын

    Me he pasado hace 1 mes

Келесі