A case study on freshness based scoring for fresh information retrieval

Abstract

For most businesses, fresh information retrieval is very important. However, it is difficult for conventional search engines based on centralized architecture to retrieve really fresh information, because they take a long time to collect documents via Web robots. In contrast to a centralized architecture, a search engine based on a distributed architecture does not need to collect documents, because each site independently makes an index. As a result, distributed search engines can retrieve really fresh information. However, fast indexing is not enough to easily retrieve fresh information. The value of information is determined by both freshness and relevance. Traditional ranking methods consider either freshness or relevance; so, we proposed FTF-IDF (fresh term frequency multiplied by inverse document frequency) as a scoring method that considers both freshness and relevance. In this paper, we describe a verification of FTF-IDF on an actual Web diary.