Method and device of determining webpage clustering mode

Abstract

The invention discloses a method and a device of determining a webpage clustering mode. The method comprises the steps of obtaining a main prefix of a uniform resource locator of a webpage to be clustered; segmenting the main prefix to obtain a plurality of fields; segmenting in a matching way to obtain a plurality of fields according to at least one preset reserve field in a reserve field dictionary and the position information of each preset reserve field, taking the field parts which are matched and identical with the preset reserve field and are provided with corresponding positions as the reserve fields, and generating the clustering mode of the uniform resource locator of the webpage to be clustered according to the reserve fields of a plurality of fields and the position information of the reserve fields. The invention also discloses a device used for realizing the method. According to the technical scheme, more webpages can be clustered under the clustering mode, the clustering effect of the webpages can be effectively optimized, and the clustering efficiency of the webpages is improved.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a method and a device for determining a web page clustering mode. Background technique [0002] In the data mining of web pages, in order to effectively mine and analyze the data of web pages, clustering technology is usually used to gather together the Uniform Resource Locator (URL) that is similar in appearance or function to form a collection. , the collection can be represented by a pattern pattern. [0003] In a web page clustering method commonly used in the prior art, URLs are segmented according to a certain delimiter to form a pattern. For example, the following URL "http: / / www.360.cn / weishi / index.html" can be divided into http: / / www.360.cn / and http: / / www.360.cn according to the separator " / ". / weishi / two prefix patterns, commonly used separators can include " / ", "?", "-" or "_" and so on. This pattern can be called prefix pattern clustering technology. The fe...

AI Technical Summary

Problems solved by technology

[0004] However, in the above-mentioned existing prefix pattern clustering technology, because the prefix pattern clustering is relatively fine, some similar webpages cannot be clustered into the same pattern

If there are two user pages http: / / 360.cn / xxx / album / and http: / / 360.cn / yyy / album / , where "xxx" and "yyy" are the identities registered by the two users respectively (Identity; ID), then the prefix clustering technology can only be clustered to http: / / 360.cn / xxx / and http: / / 360.cn / yyy / , but in fact these two user pages They are similar pages (for example, they are all user's photo albums), and cannot be clustered into the same pattern. Therefore, the existing web page clustering technology has poor clustering effect and low efficiency

Images