Video content and content source crawling method

Abstract

The invention discloses a video content and content source crawling method. The basic principle of the method is as follows: firstly setting an initial crawling period for each type of video contents, crawling and caching contents from a content source periodically, then dynamically adjusting the crawling period according to content heat and content update rate, and when the content heat or the update rate rises, shortening the crawling period; otherwise, when the heat or the update rate falls, prolonging the crawling period; and the classification basis of the ''each type of video contents'' is content metadata, including but not limited to content topic, content name, content label, etc. By implementation of the video content and content source crawling method, effectiveness of content crawling flow is improved, and the hit rate of content requests is improved. The technical scheme of the invention effectively crawls contents from a plurality of content sources (such as Internet video websites), locally caches the contents, provides video services for local users, and under a circumstance of not increasing crawling flow, improves the hit rate of the content requests.

Description

technical field [0001] The invention relates to the field of the Internet, in particular to a method for crawling or cache replacement of Internet content, and in particular to a method for crawling video content and content source. Background technique [0002] With the increasing growth of multimedia services on the Internet, the reasonable, efficient, and rapid deployment of multimedia content from the central content management system to the edge content delivery system has become a major challenge for current content operations. [0003] In order to improve the rapid release and response of multimedia content, the existing method is to adopt CDN or P2P technology. CDN deploys content to service nodes close to users, and uses technologies such as caching and redirection to provide users with efficient services when users request content. P2P is to provide services to other clients with the existing content of the client. However, these two technologies are limited by t...

AI Technical Summary

Problems solved by technology

However, these two technologies are limited by the number of nodes or users. In the case of a small number of nodes or users, the two technologies basically cannot accelerate

[0004] And the Chinese No. CN200810104585 patent discloses a method for publishing digital media content on the Internet. Although the method provides a reliable and efficient content publishing method, it focuses on reliability, and the disclosed content publishing method does not Consider the client's request, which is completely different from the method disclosed in this patent

Images