Focused Web Crawling Algorithms

Abstract

Nowadays the web is rich of any kind of information. And this information is freely available thanks to the hypermedia information systems and the Internet. This information greatly influenced our lives, our lifestyle and way of thinking. A web search engine is a complex multi-level system that helps us to search the information that available on the Internet. A web crawler is one of the most important parts of the search engine. It's a robot that systematically browses and indexes the World Wide Web. A focused web crawler is used crawling only web pages that are relevant to the user given topic or web page link. A focused crawler is a part of the search system that helps user to find most relevant information from the Internet. In our days, this area of computer science is very popular and important for the development of science and technology, because The Internet is constantly and rapidly growing, and the information in it. In this article we will review most effective focused web crawling algorithms that determines the fate of the search system.