Video recommendation method based on multi-modal video content and multi-task learning

Abstract

The invention discloses a video recommendation method based on multi-modal video content and multi-task learning. The method comprises the following steps: extracting visual, audio and text features of a short video through a pre-trained model; fusing the multi-modal features of the video by adopting an attention mechanism method; learning feature representation of the social relationship of the user by adopting a deep walk method; proposing a deep neural network model based on an attention mechanism to learn multi-domain feature representation; embedding the features generated based on the above steps into a sharing layer as a multi-task model, and generating prediction results through a multi-layer perceptron. According to the method, the attention mechanism is combined with the user features to fuse the video multi-modal features, so that the whole recommendation is richer and more personalized; meanwhile, because of multi-domain features and with consideration of the importance ofinteraction features in recommendation learning, a deep neural network model based on an attention mechanism is provided, so that learning of high-order features is enriched, and more accurate personalized video recommendation is provided for users.

Description

technical field [0001] The invention relates to the technical field of network video and recommendation systems, in particular to a video recommendation method based on multi-modal video content and multi-task learning. Background technique [0002] With the rapid popularization of smart mobile terminals and the development of multimedia technology, video has gradually become the carrier of information dissemination. In recent years, short videos have risen rapidly. Video has become a main way of entertainment for people, and users' interests have also shown wider. The rapid increase in the number of short videos has brought about a serious problem of information overload. How to find videos of interest to users from massive amounts of data has become a hot topic and research object. A good recommendation system can not only help consumers find interesting or even potentially interesting videos faster and more conveniently, but also help content providers increase profits an...

AI Technical Summary

Problems solved by technology

[0003] At present, the short video recommendation technology is facing two important challenges: (1) Most of the current recommendation algorithms are based on user preferences and user behavior to make recommendations, ignoring the content of items, and there is also a serious cold start problem, which leads to large Most videos are ignored, and even traditional content-based recommendation methods do not perform well because they rely on metadata rather than original video content

However, the metadata of micro-videos is uploaded by users, which may be inaccurate for videos, how to effectively utilize the multi-modal information of videos becomes an important challenge for video recommendation

(2) The single-task recommendation model cannot meet the current needs for multi-tasks. In video recommendation, it is not only necessary to predict whether the user will watch the video, but also predict the user's rating of the video, whether to like it, whether to forward it, etc.

Images