Multi-modal image-text recommendation method and device based on deep learning

Abstract

The invention discloses a multi-modal image-text recommendation method and device based on deep learning. The method comprises the following steps: using a cross-modal image-text retrieval model MMDNN; secondly, the MMDNN is used in a recommendation system, and a positive and negative feedback clustering center of the user is calculated by using a positive and negative feedback clustering center calculation module PNFCCCM and positive and negative feedback historical records of the user; in combination with the similarity score and the positive and negative feedback score of the data, searching several pieces of data with the highest comprehensive score in the historical records of the user are from the database, and using the MMDNN model for finding out data of another mode corresponding to the several pieces of data from the database, and finally, recommending paired graph-text resources to the user; and updating the historical record of the user and the positive and negative feedback clustering center of the user according to the feedback of the user, so that multi-modal image-text recommendation is realized.

Description

technical field [0001] The invention belongs to the field of computer science and technology application, and in particular relates to a deep learning-based multimodal graphic-text recommendation method and device. Background technique [0002] Currently, most recommender systems focus on providing a single mode of content, such as using images to recommend images and using text to recommend text. In fact, images and text resources in different forms have an unbalanced and complementary relationship when describing the same semantics. Images can usually contain more details that cannot be displayed in text, and text has the advantage of expressing high-level meaning. Therefore, users need more multimodal information resources, and cross-modal retrieval technology is more concerned. Cross-modal retrieval is a technology that can return information combined with multiple modes according to the information entered by the user in one mode. At present, many cross-modal retrieva...

AI Technical Summary

Problems solved by technology

At present, many cross-modal retrieval methods are only applied in the field of retrieval, and have not been applied to the field of recommendation system.

Moreover, these cross-modal retrieval methods have shortcomings such as insufficient retrieval accuracy and long time consumption.

Images