Data processing method and device, computer system and storage medium

Abstract

The invention provides a big data processing method and device, a computer system and a storage medium. The method comprises: storing data in a database; extracting unstructured data and missing datain the database; sorting the unstructured data into structured data, complementing and processing missing data to obtain processed data, constructing a factor library from the processed data, performing detrending operation and standardization operation on the factor library, and inputting factors in the factor library into a training model for operation to obtain a model training result. According to the data processing method and device, the computer system and the storage medium provided by the invention, the collected unstructured data in the database is sorted into the structured data, and the missing data is subjected to supplementary processing, so that errors caused by the missing data are prevented, and non-systematic data input errors are reduced. In addition, the invention alsorelates to a block chain technology, and the obtained model training result can be stored in the block chain node.

Description

technical field [0001] The present invention relates to the technical field of big data processing, in particular to a data processing method, equipment, computer system and storage medium. Background technique [0002] In the application of computer technology, it is often necessary to use artificially processed data as input, which will rely on a large number of human resources, and the information and data required in the work are huge and complicated, and the data sources are also very extensive, but the existing The system often can only read the information data in the existing reports. For other information that needs simple judgment and processing, it cannot be collected because it has not been processed into formatted data information. It can only be manually summarized after collection. Entry system. If more data requires manual input and judgment, the efficiency and coverage of the system will be greatly reduced. When new information appears in the work, unless m...

AI Technical Summary

Problems solved by technology

[0002] In the application of computer technology, it is often necessary to use artificially processed data as input, which will rely on a large number of human resources, and the information and data required in the work are huge and complicated, and the data sources are also very extensive, but the existing The system often can only read the information data in the existing reports. For other information that needs simple judgment and processing, it cannot be collected because it has not been processed into formatted data information. It can only be manually summarized after collection. Entry system

If more data requires manual input and judgment, the efficiency and coverage of the system will be greatly reduced. When new information appears in the work, unless manual intervention completes the information, it will not be possible to give feedback on these new information

[0003] Especially when various data are applied to credit rating, traditional rating methods often require a lot of information to achieve, and usually require a lot of human resources to simply process data as input. Due to the diversity of information sources, data processing is particularly important, and ratings require The information and data are huge and complicated, and the data sources are also very extensive, which will also cause inconvenience in data ingestion. The extraction of effective information from various data makes data processing complicated. Accuracy requires summarizing opinions after information collection, and then manually entering the system, which makes the processing process longer and takes more time, and cannot meet the current requirements for information processing

Images