Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Global and local feature embedding and image-text fusion sentiment analysis method and system

A local feature and sentiment analysis technology, which is applied in the field of sentiment analysis of social media graphics and texts, can solve problems such as insufficient emotional features and inability to accurately judge user emotions, and achieve the effect of enhancing expression and improving accuracy

Active Publication Date: 2020-02-28
NANJING UNIV OF POSTS & TELECOMM
View PDF4 Cites 27 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method combines the global and local features of the image to jointly mine the emotion of the image, further improves the accuracy of image emotion recognition, and integrates the emotional information of the text to solve the problem that the global emotional features of the image are not rich enough and the emotional features of a single image text cannot accurately judge the user's emotion. problem, improving the accuracy of social media sentiment analysis

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Global and local feature embedding and image-text fusion sentiment analysis method and system
  • Global and local feature embedding and image-text fusion sentiment analysis method and system
  • Global and local feature embedding and image-text fusion sentiment analysis method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] The technical solution of the present invention will be described in detail below in conjunction with the drawings:

[0037] Such as figure 1 As shown, the embodiment of the present invention discloses a sentiment analysis method of global and local feature embedding and image-text fusion. The convolutional neural network is used to extract the global features of the image, and the target detection data set is used to pre-train the target detection model through migration learning. Use the emotional image data set to train the target detection model again, detect and locate the effective target area carrying emotion in the image, extract the local area features detected by the detection, and then embed the extracted local area features into the deep features extracted from the image globally To jointly train the image emotion classification model to obtain the emotion polarity probability of the image. Then through training the word vector model, the text is expressed as ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a global and local feature embedding and image-text fusion sentiment analysis method and system. The method comprises the following steps: firstly, extracting global features of an image by using a convolutional neural network, training a target detection model by using transfer learning, detecting and positioning a local region carrying emotion in the image, extracting local region features, and embedding the local region features into the global features to jointly train an image emotion classification model to obtain an emotion polarity probability of the image; expressing the text as a word vector containing rich semantic information, and inputting the word vector into a bidirectional LSTM capable of extracting text semantic context features to perform sentimentclassification to obtain the polarity probability of sentiment of the text; and finally, according to a later fusion formula, obtaining an emotion polarity probability after image and text fusion, and performing image-text emotion analysis. The method can effectively pay attention to the emotion information of the image-text, and improves the accuracy and robustness of image-text emotion classification through the extraction of global and local emotion features of the image and the fusion of text information.

Description

Technical field [0001] The present invention relates to an emotion analysis method and system for global and local feature embedding and image and text fusion, in particular to the emotion analysis of social media images and text, and belongs to the technical field of emotion recognition intersected by computer vision image and natural language processing. Background technique [0002] The sentiment analysis of images and text is a research topic involving computer vision, pattern recognition, and natural language processing. With the continuous development of the Internet and the maturity of social media, more and more Internet users like to share their experiences on social platforms such as Weibo and WeChat and participate in discussions on various events and topics. Social networking sites have become Internet users An important platform for opinions and feelings. By analyzing the opinions and pictures shared by users on social media, mining users’ emotional tendencies can b...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/62G06K9/46G06F40/30G06N3/04G06N3/08
CPCG06N3/084G06V10/44G06N3/048G06N3/044G06N3/045G06F18/241
Inventor 刘天亮林明亮戴修斌谢世朋
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products