AI-based objectified attribute text automatic classification method and system

Abstract

The invention belongs to the technical field of data analysis and data mining, and particularly relates to an AI-based objectified attribute text automatic classification method and system, and the method comprises the following core steps: building a character coding library, carrying out character decomposition on all text data which are stored in a library in history, numbering single characters uniquely in the library, and integers being used as self-increasing numbers according to numbering rules; preprocessing standard attribute data, extracting stored standard data as to-be-trained data, and limiting the length of a character string to be 60 Chinese characters, such as drug general names, drug specifications, drug production enterprises, approved numbers and the like, and clearly expressing fields of data attribute characteristics. Through the invention, the subject attribute category described by a section of data can be quickly judged, and then whether the attribute category is consistent with the subject design or not is judged; in addition, attribute classification judgment can be carried out on a plurality of adjacent data, and the position of the main body description information can be positioned in the webpage.

Description

technical field [0001] The present invention relates to the technical field of data analysis and data mining, in particular to an AI-based automatic classification method and system for object-oriented attribute texts. Background technique [0002] In just five years, the number of people using the Internet has increased by 83%. Taking Weibo as an example, the monthly active users of Weibo increased to 462 million at the end of 2018, and the average daily text publishing volume reached 130 million. In the face of massive amounts of data, simple manual management and induction of different types of information will cost a lot of time and money. More and more applications are adopting automatic text classification technology, including spam comment recognition, pornography recognition, news classification, sentiment analysis, etc. Text classification technology is in a period of rapid development under the background of big data. [0003] After searching, the Chinese patent ...

AI Technical Summary

Problems solved by technology

[0004] The above-mentioned patent also has the following disadvantages: in most algorithms of machine learning, it is not possible to directly use text as a feature value for training. In the current feature engineering description, it does not explain how to calculate the similarity and value between texts. In this way, it is difficult to achieve the specified goal through model training; the algorithm mentions "manually labeled real label data" and uses it as positive sample data for training. If you want the results to be ideal, you must go through a lot of manual labeling. Yes, this kind of manual workload is huge, and it takes a lot of time to implement the algorithm. If there is a new classification situation, manual participation is required, and it takes the same or more time to label, which is obviously not very realistic;

Images