Research for framework of subject deep web crawler
Yang Hu
Abstract
Yang Hu
Abstract
To satisfy people's demand for getting precise and personal information,characteristics of deep web information are analyzed,and a framework of crawler for searching different subject information in deep web is put forward. To solve the difficult problems of deep web database discovery and deep web crawler crawling strategy,the technologies of discovering different subject deep web database quickly to use the universal search engine and downloading deep web information to the utmost by adopting the commonly used Chinese characters are proposed respectively. At last the experiment show that the framework is correct,and the technologies are feasible.
A significance statement is not available in the OpenAlex record.
A contribution statement is not available in the OpenAlex record.
Method details are not available in the OpenAlex metadata.
Findings are not separately available in the OpenAlex metadata.
Limitations are not available in the OpenAlex metadata.
Application details are not available in the OpenAlex metadata.
To satisfy people's demand for getting precise and personal information,characteristics of deep web information are analyzed,and a framework of crawler for searching different subject information in deep web is put forward. To solve the difficult problems of deep web database discovery and deep web crawler crawling strategy,the technologies of discovering different subject deep web database quickly to use the universal search engine and downloading deep web information to the utmost by adopting the commonly used Chinese characters are proposed respectively. At last the experiment show that the framework is correct,and the technologies are feasible.
Key concepts: Web crawler, Computer science, Deep Web, Crawling, Focused crawler, Upload, World Wide Web, Subject (documents)