Hadoop-based network recommendation method

Abstract

The invention discloses a Hadoop-based network recommendation method. Specific to a network based on a Hadoop platform, the scale of a cluster is increased according to a practical computation requirement; a dynamic storage space is provided for mass data sets by using an HDFS (Hadoop Distributed File System) to store the ever-increasing mass data sets; big tasks are decomposed into small tasks through a MapReduce framework; the computation speed of an algorithm is increased through a parallel computation way; and network recommendation is realized by using a substance diffusion algorithm and a heat conduction algorithm. Through adoption of the Hadoop-based network recommendation method, the problem that a large electronic commerce website cannot be well reached since a conventional algorithm running in a standalone mode tends to be impeded easily by multiple computer hardware performance such as processor speed and storage capacity is solved; a system is reliably and efficiently recommended to perform calculation, analysis and mining through massive user dialogs; and the problems of high time consumption, poor expandability, polynomial increase of the complexity of the algorithm and worse and worse algorithm performance along with the increase of the numbers of users and goods, and the like in other recommendation algorithms are solved.

Description

technical field [0001] The invention relates to the field of recommendation methods, in particular to a hadoop-based network recommendation method. Background technique [0002] With the rapid innovation of Internet technology, the total number of web servers in the world continues to increase, and the number of web pages therefore shows an explosive growth trend. People can read articles, listen to music, communicate socially, share photos, and buy goods online, but it also brings a series of troubles: on the one hand, it is difficult for users to search for valuable parts from the explosive accumulation of massive information; On the one hand, the quality of information itself varies, and some potentially valuable information is buried due to the lack of auxiliary tools. This phenomenon is called "information overload". In order to effectively deal with the above problems, people have developed search engines, such as Google, Baidu, Bing, etc., which use web crawlers to c...

AI Technical Summary

Problems solved by technology

In modern e-commerce websites, response speed is one of the important indicators of web application user experience, but the poor scalability of the recommendation system and the consumption of a large amount of computing time are intolerable. How can the recommendation system recommend products to users in a short time? With tens of thousands of online users, this is a serious challenge

Images