Data preprocessing method and device, computer equipment and storage medium

Abstract

The invention relates to a data preprocessing method and system, computer equipment and a storage medium. The method comprises the following steps of: obtaining at least one sampling data table and obtaining data stored in at least one field of the sampling data table; obtaining a mapping relationship between fields in a data model table and pre-configured table fields, wherein the mapping relationship comprises a one-to-one or one-to-many relationship between the fields in the data model table and the fields in the sampling data table; mapping the data stored in the fields of the sampling data table into corresponding fields of the data model table according to the mapping relationship so as to obtain a data-mapped data model table; and storing the data-mapped data model table. Accordingto the method, when business analysis is carried out on different pieces of bottom data, data screening does not need to be carried out repeatedly, and data cleaning can be automatically realized through adjusting the mapping relationship, so that the data processing efficiency is improved.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a data preprocessing method, device, computer equipment and storage medium. Background technique [0002] With the increasing demands of users and the diversified development of business, when doing data analysis for some enterprises, such as data consulting projects such as user grouping and precision marketing for banks, it is usually necessary to analyze the data provided by the bank. To perform multi-layer screening, firstly, the underlying data needs to be cleaned, followed by steps such as standardization of the cleaned data, index development, modeling analysis, etc. Each step is customized according to the characteristics of each bank, and the longest time-consuming and The most cumbersome work is the underlying data cleaning and indicator development. The reason is that the underlying data storage form of the bank is mainly for the bank to carry out bus...

AI Technical Summary

Problems solved by technology

[0002] With the increasing demands of users and the diversified development of business, when doing data analysis for some enterprises, such as data consulting projects such as user grouping and precision marketing for banks, it is usually necessary to analyze the data provided by the bank. To perform multi-layer screening, firstly, the underlying data needs to be cleaned, followed by steps such as standardization of the cleaned data, index development, modeling analysis, etc. Each step is customized according to the characteristics of each bank, and the longest time-consuming and The most cumbersome work is the underlying data cleaning and indicator development. The reason is that the underlying data storage form of the bank is mainly for the bank to carry out business services, which is quite different from the requirements of data modeling and analysis. It needs to be manually screened out. Useful data for analysis is aggregated into statistical indicators required for modeling and analysis. Every time data analysis is performed on another bank, it takes a lot of time to carry out this customized work

Images