A company industry classification calculation method based on natural language processing

Abstract

The invention discloses a company industry classification calculation method based on natural language processing. Text data of a pre-classification company are obtained through a crawler, feature extraction, noise reduction processing and word vector training are conducted on the text data, hierarchical classification is conducted on the text data after a language model and a transfer learning pre-training classification model are adopted, and classification of a target company is achieved. The method is simple in process, high in efficiency and capable of saving manpower and material resources; according to the method, about 30 primary classifications and about 300 secondary classifications can be obtained through a hierarchical classification system, so that the classification accuracyis greatly improved; the model can accept text input of different lengths and forms, the model does not need to be adjusted at all, the application range is wider, and the practicability is higher.

Description

technical field [0001] The present invention relates to the field of Internet technology, in particular to a method for calculating company industry classification based on natural language processing. Background technique [0002] In data search, accurate industry positioning can help users quickly judge whether the target company meets their needs. The existing industry classification mainly uses manual methods to mark the company's industry category, formulate industry classification rules to judge the company's industry or traditional classification methods (such as support vector machines / decision trees, etc.), there are the following problems: [0003] (1) Manual method: There are knowledge barriers among various industries, and a large number of industry experts are required to effectively complete the labeling, which consumes a lot of manpower and material resources; [0004] (2) Rule method: the number of companies is huge, it is difficult to formulate industry cla...

AI Technical Summary

Problems solved by technology

[0003] (1) Manual method: There are knowledge barriers among various industries, and a large number of industry experts are required to effectively complete the labeling, which consumes a lot of manpower and material resources;

[0004] (2) Rule method: the number of companies is huge, it is difficult to formulate industry classification rules taking into account the characteristics of all companies; and new companies emerge in endlessly, it is difficult to update in time; at the same time, the formulation of rules requires the participation of a large number of people, which is difficult to implement;

[0005] (3) Traditional classification methods: feature extraction processing is required, and the document loss information after processing may easily lead to a decrease in classification accuracy