Webpage ranking method based on cloud computation

Abstract

The invention discloses a webpage ranking method based on cloud computation. The method comprises the following steps of analyzing a network file which is crawled by a distributive webpage crawler to obtain a basic topological structure information file of a network; offline calculating a PR value and then storing the PR value into a corresponding document table, wherein the format of the document table adopts url as a main key , and the format containing eight attribute columns containing title, content, type, timestamp, outlinks and the like; adopting a parallel computation technology for establishing an index table of single word - webpage importance, wherein the format of the index table is a format established by a reverse index and containing key and links (link set and sorted according to the importance; adopting a MapReduce parallel architecture to realize the offline PageRank algorithm; and comparing the similarity of an inquiry word and a webpage for online inquiry, and giving a final webpage rank according to the offline inquiry result. The method has the advantages that the offline ranking algorithm is adopted, the MapReduce parallel arhictecture is adequately utilized, so that the offline ranking efficiency is improved; and by adopting the technology combining the key word technology and the PageRank technology, the result is more accurate.

Description

Technical field [0001] The invention involves a distributed web sorting method, especially the webpage sorting method based on cloud computing under big data. Background technique [0002] With the rapid development of Internet, WWW (World Wide Web referred to as www) has become a huge information space, providing users with valuable information resources.In the face of a large amount of information resources, it is very inconvenient to browse by browser step by step. How to get the required information from WWW quickly and accurately has become a crucial issue.The emergence of search engines has greatly improved the ability to collect information.However, existing search engines still have problems and difficulties in search efficiency, information maintenance, information repetition, network and sites, loads, etc. [0003] At present, from the perspective of architecture, most search engines are concentrated.That is, take the page from the Internet. After analysis and processin...

AI Technical Summary

Problems solved by technology

[0006] (1) The problem of massive network data: the amount of network information is large and the coverage is wide, and the calculation and storage of these data need to consume a lot of time and storage space

[0007] (2) Response time problem during online sorting: Since a large amount of webpage information will be retrieved according to keywords, the response time of search engines is relatively long, which does not meet the requirements of search engines for response time

[0008] (3) The accuracy of webpage sorting: the accuracy of webpage sorting is not only related to keywords, but also has an important relationship with the quality of the webpage itself

[0009] (4) Expression of search terms: Due to the lack of domain knowledge of users and the limitations of the query interface of search engines, it is impossible to accurately realize the user's search intent

Images