Social media data processing method, model training method and related devices

Abstract

The embodiment of the invention relates to the technical field of natural language processing, and provides a social media data processing method, a model training method and related devices. The method comprises the steps: preprocessing obtained social media data into a plurality of to-be-recognized sequences; inputting each sequence to be identified into the trained entity recognition model; themethod comprises the steps of obtaining an embedded sequence of a to-be-identified sequence by utilizing an embedded layer, then inputting the embedded sequence into a dynamic neural network layer for multiple rounds of feature extraction and sequence prediction to obtain all entity sequences output by the dynamic neural network layer, and obtaining all basic entities and nested entities in the to-be-identified sequence according to all the entity sequences; and finally, obtaining a processing result corresponding to the social media data according to the entity results corresponding to all the to-be-identified sequences. Compared with the prior art, all the basic entities and the nested entities in the social media data can be obtained, and then people are helped to quickly know the latest dynamic and hot events of the current society.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of natural language processing, and in particular, relate to a social media data processing method, a model training method, and related devices. Background technique [0002] With the advent of the information age, various social media have sprung up, for example, social networking sites, Weibo, WeChat, blogs, forums, encyclopedias, etc. People are becoming more and more accustomed to instantly obtaining and sharing information through various social media. Social media generates massive amounts of data every day, involving various fields such as politics, economy, military affairs, entertainment, and life. Analyzing these data can help people quickly perceive social hotspots. [0003] Named Entity Recognition (Named Entity Recognition) refers to the recognition of proper names such as person names, place names, organization names, and meaningful time, date, and other quantitative phr...

AI Technical Summary

Problems solved by technology

Existing methods can better identify basic entities, but it is difficult to identify complete nested entities at the same time, resulting in insufficient recognition accuracy

Images