The other Kademlia
A look at "Sloppy hashing and self-organizing clusters" by Michael J. Freedman and David Mazières (2002). These are the same authors of the Kademlia DHT paper
00:00 Sloppy hashing and self-organizing clusters
01:22 DHTs are the wrong Abstraction
02:31 DHTs have poor locality
04:30 Le Network
06:01 Kademlia Example
08:47 Clustering
13:50 Sloppiness
16:17 Self-organizing clusters
17:04 Cluster Discovery
19:52 Cluster Merging & Splitting
21:41 Small Privacy Concerns
22:12 Implementation Challenges
23:40 Closing Thoughts
Пікірлер: 18
It is awesome that you do these videos on such a niche topic. They are exactly what I was looking for.
I too have realized that the Kademlia topology has some challenges - e.g. with regards to broadcast / multicast, geographical co-location of nodes, only symmetric scalability etc. To solve these challenges I designed a P2P topology which I call the "Polymorph Polyring" topology. A polyring consists of a central ring of peers, where each peer can have a sub-ring attached, and each peer in a sub-ring can again have a sub-ring attached - recursively - indefinitely. This topology enables peers that are close to communicate quickly, while keeping their data within their sub-ring (which could be located within the same country, region, city neighbourhood etc.). It also makes broadcast and multicast a lot easier, as there is only 1 path to each node in the whole network. No double-receivals of messages - like gossipy protocols have. Each ring (central + sub-rings) can be scaled in size independently of any other rings, so you can do assymetric, local scaling. I am also working on how to implement a DHT on top of the Polymorph Polyring topology. I have some core ideas for that now - but it's not yet finished 🙂
@n0computer
Ай бұрын
Well do a video on HyParView & PlumTree at some point, but it’s our current favorite for building topologies that optimize for latency
Great video, great production quality 👏
@n0computer
Жыл бұрын
Thanks!
💡
@n0computer
Жыл бұрын
It’s exciting! Complicated, but exciting ☺️
Does Node D (in this example) have to maintain an entry for all nodes inside its cluster? That is to say, if it's seeing F, but D doesn't have an entry, how does it know if it needs to cascade to the lower level cluster or not?
Great video. Just one question: Why not simply use plain kademlia and priorize the k-bucket entries by local round drip delay per bucket? In the end this would create k-buckets with the buckets sorted by node ID's similarity, but the individual bucket entrys would be priorized by round drip delay. A new node belonging to a full bucket would always end up replacing an entry in that bucket if it has a lower latency.
@louisthibault555
8 ай бұрын
The entries in your k buckets are not guaranteed to be nearby. The Coral DHT (being discussed here) is good at making sure physically-nearby machines are over-represented in the routing table.
I am searching to make a p2p media system to fight censorships. So having a protocol which makes it difficult to get where are the nodes is good. I think that putting rastreability in p2p goes against its own premise
Great explanation!
@n0computer
Жыл бұрын
Thanks!
What sw is used to show network topology?
@n0computer
Ай бұрын
It’s a vector drawing done by hand (using adobe illustrator in this video)
@LukeMahan-xr4xx
Ай бұрын
@@n0computer thx
Hi! Is there a way to get in contact with you somehow?
@n0computer
Ай бұрын
iroh.computer/discord