Personalized recommendation method and system based on improved clustering and Spark framework

Abstract

The invention discloses a personalized recommendation method based on improved clustering and a Spark framework. The personalized recommendation method comprises the steps of determining an effectivescoring data set; performing clustering preprocessing on the project by utilizing a Canopy algorithm to generate at least one Canopy clustering center; initializing a clustering center of the FCM algorithm, updating the membership degree of each project to the clustering center by using a membership degree calculation formula, updating the clustering center according to the updated membership degree, and performing iteration until a stop condition is met, and determining a final clustering set; respectively calculating the similarity between the target project and each clustering center in thefinal clustering set, selecting projects in the clustering set corresponding to the similarity greater than or equal to a preset similarity threshold to form a candidate project space, calculating the similarity between the target project and each project in the candidate project space, and searching a K nearest neighbor set of the target project; obtaining a preference prediction value of the user for the target project, and using the top-N recommendation method to select N items with higher preference prediction values for recommendation..

Description

technical field [0001] The present invention relates to the field of personalized recommendation of big data, and more specifically, relates to a personalized recommendation method and system based on improved clustering and Spark framework. Background technique [0002] The rapid development of the mobile Internet marks the entry of human beings into the era of big data. Serious information overload makes it difficult for users to obtain the required information conveniently. In this context, personalized recommendation technology emerges as the times require. Collaborative filtering is one of the most successful and widely used recommendation techniques, mainly divided into two categories: user-based collaborative filtering and item-based collaborative filtering. [0003] Since the update speed of items is relatively slow and the number of items is much smaller than the number of users, the cost of calculating item similarity is far less than that of calculating user simil...

AI Technical Summary

Problems solved by technology

[0006] The present invention provides a personalized recommendation method and system based on improved clustering and Spark framework to solve the problem of how to provide users with personalized recommendations quickly and accurately

Images