Yu Hongbo
Abstract
As an indispensable part of search engine,web crawler is used to download web pages from Internet.Classifications and principles of web crawler are introduced.The topic crawler is designed in detail,and then technical problems of designing high-performance web crawler to be solved are summarized.
OpenAlex reports 6 citations for this work. Citation counts describe recorded attention and do not establish research quality.
A contribution statement is not available in the OpenAlex record.
Method details are not available in the OpenAlex metadata.
Findings are not separately available in the OpenAlex metadata.
Limitations are not available in the OpenAlex metadata.
Application details are not available in the OpenAlex metadata.
As an indispensable part of search engine,web crawler is used to download web pages from Internet.Classifications and principles of web crawler are introduced.The topic crawler is designed in detail,and then technical problems of designing high-performance web crawler to be solved are summarized.
Key concepts: Web crawler, Focused crawler, World Wide Web, Computer science, Web search engine, Web page, Information retrieval, The Internet