Index Sieving - A Fast Ranking Search Method for Large-scale Search Engines

Most of today's document retrieval systems use inverted indices to implement fast and efficient search. However, as the targeted document collection is getting large, postings read from the inverted index also become large and search speed is sacrificed. In this paper, we propose index sieving, a method of reducing the size of postings read from the index to accelerate search. With index sieving, we first search highly relevant documents using an index which stores postings that will largely contribute to relevance scores. Effectiveness of the proposed method is evaluated by experiments using a large number of web pages and queries to a real web search engine. Keyword Inverted Index, Information Retrieval, Search Engine, Performance Evaluation