Web crawler service system for housing library network

Abstract

The invention provides a web crawler service system for a housing library network. According to the system, website mining can be conducted quickly, and data related to housing estates can be extracted. The system comprises a website crawler module, a monitoring service module, a management service module, a deployment service module and a scheduling service module, wherein the website crawler module consists of multiple website crawlers, the website crawlers are in one-to-one correspondence with websites, analyze page elements of the websites, extract website data for semantic analysis and map the website data into preset data entities for storage; the monitoring service module is used for monitoring working conditions of all the website crawlers and judging whether the website crawlers work normally or not and whether data fetching is correct or not; the management service module is used for configuring settings of parameters related to working of the website crawlers, upgrading the website crawlers and managing start-up and stopping of the service system as well as life cycles and working of the website crawlers; the deployment service module is used for allocating and deploying the website crawlers; scheduling modules of the website crawlers are arranged in the scheduling service module and used for performing scheduling management on working modes, time and stopping of the website crawlers.

Description

technical field [0001] The invention relates to the technical field of website data mining, in particular to a web crawler service system for Fangku.com. Background technique [0002] The real estate industry is directly related to people's livelihood. The current residential market will enter the era of stock houses, and many home owners of stock houses are not professional salespersons, and the sales information provided is not comprehensive enough. At the same time, the file management of housing in government departments is still at the paper stage, and various data related to housing and real estate are scattered in various units and departments. This kind of effective data is not fully utilized. People choose housing, enterprises choose office space will face a serious lack of professional and detailed information services. [0003] In the social environment, it is of great significance to promote real estate informatization, facilitate home buyers to query informat...

AI Technical Summary

Problems solved by technology

The foundation of building a database is data mining, but in today’s era of high-speed information dissemination and spam flying everywhere, how to quickly and effectively carry out website data mining has always been a hot topic, and no ideal and effective solution has been found.

Images