Multi-modal scene recognition method based on deep learning

Abstract

The invention discloses a multi-modal scene recognition method based on deep learning. The multi-modal scene recognition method comprises the following steps: S1, carrying out word segmentation processing on a short text; S2, inputting a group of pictures, short text segmented words and corresponding tags into respective convolutional neural networks for training; S3, training a short text classification model; S4, training a picture classification model; S5, respectively calculating cross entropies of the full connection layer outputs in S3 and S4 and a standard classification result, calculating an average Euclidean distance which serves as a loss value, then feeding back the loss value to the respective convolutional neural network, and finally obtaining a complete multi-modal scene recognition model; S6, adding the text and the image prediction result vector to obtain a final classification result; and S7, respectively inputting the short text and the image to be identified into the trained multi-modal scene identification model, and performing scene identification. The invention provides a multi-modal scene searching mode, and more accurate and convenient scene recognition isprovided for users.

Description

technical field [0001] The invention relates to a multimodal scene recognition method, in particular to a deep learning-based multimodal scene recognition method, which belongs to the fields of artificial intelligence and pattern recognition. Background technique [0002] Deep learning is a brand-new field of machine learning. Its purpose is to make machine learning closer to human intelligence. Convolutional neural network is a representative algorithm of deep learning. It has the characteristics of simple structure, strong adaptability, few training parameters and many connections. , therefore, this network has been widely used in the fields of image processing and pattern recognition for many years. [0003] Specifically, the convolutional neural network is a hierarchical model whose input is the original data. Through a series of operations such as convolution operations, pooling operations, and nonlinear activation functions, the high-level semantic information is layer...

AI Technical Summary

Problems solved by technology

The results obtained by the method of feature fusion are more objective, but the actual implementation method is too complicated

Images