Search engines and Web dynamics

In this paper we study several dimensions of web dynamics in the context of large-scale Internet search engines. Both growth and update dynamics clearly represent big challenges for search engines. We show how the problems arise in all components of a reference search engine model. Furthermore, we use the FAST Search Engine architecture as a case study for showing some possible solutions for web dynamics and search engines. The focus is to demonstrate solutions that work in practice for real systems. The service is running live at www.alltheweb.com and major portals worldwide with more than 30 million queries a day, about 700 million full-text documents, a

• Type: Journal
• Year: 2002
• Where: CN
• Authors: Knut Magne Risvik, Rolf Michelsen
• Keywords: CN 2002, Internet Search Engines, Search Engine, Web Dynamics