Exploiting Similarity for Multi-Source Downloads Using File Handprints

Many contemporary approaches for speeding up large file transfers attempt to download chunks of a data object from multiple sources. Systems such as BitTorrent quickly locate sources that have an exact copy of the desired object, but they are unable to use sources that serve similar but non-identical objects. Other systems automatically exploit cross-file similarity by identifying sources for each chunk of the object. These systems, however, require a number of lookups proportional to the number of chunks in the object and a mapping for each unique chunk in every identical and similar object to its corresponding sources. Thus, the lookups and mappings in such a system can be quite large, limiting its scalability. This paper presents a hybrid system that provides the best of both approaches, locating identical and similar sources for data objects using a constant number of lookups and inserting a constant number of mappings per object. We first demonstrate through extensive data ana...

• Type: Conference
• Year: 2007
• Where: NSDI
• Authors: Himabindu Pucha, David G. Andersen, Michael Kaminsky
• Keywords: Computer Networks, Constant Number, Data Objects, NSDI 2007, Similar Sources