The other Kademlia

A look at "Sloppy hashing and self-organizing clusters" by Michael J. Freedman and David Mazières (2002). These are the same authors of the Kademlia DHT paper
00:00 Sloppy hashing and self-organizing clusters
01:22 DHTs are the wrong Abstraction
02:31 DHTs have poor locality
04:30 Le Network
06:01 Kademlia Example
08:47 Clustering
13:50 Sloppiness
16:17 Self-organizing clusters
17:04 Cluster Discovery
19:52 Cluster Merging & Splitting
21:41 Small Privacy Concerns
22:12 Implementation Challenges
23:40 Closing Thoughts

Пікірлер: 18

  • @bobkelso5681
    @bobkelso5681 Жыл бұрын

    It is awesome that you do these videos on such a niche topic. They are exactly what I was looking for.

  • @JakobJenkov
    @JakobJenkov3 ай бұрын

    I too have realized that the Kademlia topology has some challenges - e.g. with regards to broadcast / multicast, geographical co-location of nodes, only symmetric scalability etc. To solve these challenges I designed a P2P topology which I call the "Polymorph Polyring" topology. A polyring consists of a central ring of peers, where each peer can have a sub-ring attached, and each peer in a sub-ring can again have a sub-ring attached - recursively - indefinitely. This topology enables peers that are close to communicate quickly, while keeping their data within their sub-ring (which could be located within the same country, region, city neighbourhood etc.). It also makes broadcast and multicast a lot easier, as there is only 1 path to each node in the whole network. No double-receivals of messages - like gossipy protocols have. Each ring (central + sub-rings) can be scaled in size independently of any other rings, so you can do assymetric, local scaling. I am also working on how to implement a DHT on top of the Polymorph Polyring topology. I have some core ideas for that now - but it's not yet finished 🙂

  • @n0computer

    @n0computer

    Ай бұрын

    Well do a video on HyParView & PlumTree at some point, but it’s our current favorite for building topologies that optimize for latency

  • @wesfloyd
    @wesfloyd Жыл бұрын

    Great video, great production quality 👏

  • @n0computer

    @n0computer

    Жыл бұрын

    Thanks!

  • @IPFSbot
    @IPFSbot Жыл бұрын

    💡

  • @n0computer

    @n0computer

    Жыл бұрын

    It’s exciting! Complicated, but exciting ☺️

  • @davidaronchick-expanso
    @davidaronchick-expanso9 ай бұрын

    Does Node D (in this example) have to maintain an entry for all nodes inside its cluster? That is to say, if it's seeing F, but D doesn't have an entry, how does it know if it needs to cascade to the lower level cluster or not?

  • @box-hu6is
    @box-hu6is10 ай бұрын

    Great video. Just one question: Why not simply use plain kademlia and priorize the k-bucket entries by local round drip delay per bucket? In the end this would create k-buckets with the buckets sorted by node ID's similarity, but the individual bucket entrys would be priorized by round drip delay. A new node belonging to a full bucket would always end up replacing an entry in that bucket if it has a lower latency.

  • @louisthibault555

    @louisthibault555

    8 ай бұрын

    The entries in your k buckets are not guaranteed to be nearby. The Coral DHT (being discussed here) is good at making sure physically-nearby machines are over-represented in the routing table.

  • @r.a.a.6472
    @r.a.a.64722 ай бұрын

    I am searching to make a p2p media system to fight censorships. So having a protocol which makes it difficult to get where are the nodes is good. I think that putting rastreability in p2p goes against its own premise

  • @marcomunizaga3022
    @marcomunizaga3022 Жыл бұрын

    Great explanation!

  • @n0computer

    @n0computer

    Жыл бұрын

    Thanks!

  • @LukeMahan-xr4xx
    @LukeMahan-xr4xxАй бұрын

    What sw is used to show network topology?

  • @n0computer

    @n0computer

    Ай бұрын

    It’s a vector drawing done by hand (using adobe illustrator in this video)

  • @LukeMahan-xr4xx

    @LukeMahan-xr4xx

    Ай бұрын

    @@n0computer thx

  • @moonlessiguana1826
    @moonlessiguana18267 ай бұрын

    Hi! Is there a way to get in contact with you somehow?

  • @n0computer

    @n0computer

    Ай бұрын

    iroh.computer/discord