On the k-Independence Required by Linear Probing and Minwise Independence

We show that linear probing requires 5-independent hash functions for expected constant-time performance, matching an upper bound of [Pagh et al. STOC’07]. For (1 + ε)-approximate minwise independence, we show that Ω(lg 1 ε )-independent hash functions are required, matching an upper bound of [Indyk, SODA’99]. We also show that the multiply-shift scheme of Dietzfelbinger, most commonly used in practice, fails badly in both applications.

• Type: Conference
• Year: 2010
• Where: ICALP
• Authors: Mihai Patrascu, Mikkel Thorup
• Keywords: 5-independent Hash Functions, Hash Functions, ICALP 2010, Programming Languages, Upper Bound