SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication

We introduce SpiderCast, a distributed protocol for constructing scalable churn-resistant overlay topologies for supporting decentralized topic-based pub/sub communication. SpiderCast is designed to effectively tread the balance between average overlay degree and communication cost of event dissemination. It employs a novel coverage-optimizing heuristic in which the nodes utilize partial subscription views (provided by a decentralized membership service) to reduce the average node degree while guaranteeing (with high probability) that the events posted on each topic can be routed solely through the nodes interested in this topic (in other words, the overlay is topic-connected). SpiderCast is unique in maintaining an overlay topology that scales well with the average number of topics a node is subscribed to, assuming the subscriptions are correlated insofar as found in most typical workloads. Furthermore, the degree grows logarithmically in the total number of topics, and slowly decrea...

• Type: Conference
• Year: 2007
• Where: DEBS
• Authors: Gregory Chockler, Roie Melamed, Yoav Tock, Roman Vitenberg
• Keywords: Average Node Degree, Average Overlay Degree, DEBS 2007, Distributed And Parallel Computing, Pub/sub Communication