Database data cleaning method and device

Abstract

The invention provides a database data cleaning method and device, electronic equipment and a computer program product, and the method comprises the steps: obtaining a data storage mapping relation between a data synchronization platform and a data storage platform, and obtaining a personalized cleaning strategy; generating a data cleaning task based on the personalized cleaning strategy and the database index information; and distributing the data cleaning task to a pre-created data cleaning process based on the data storage mapping relationship so as to perform data cleaning on the data storage platform through the data cleaning process. According to the method, the data storage mapping relation between the data synchronization platform and the data storage platform is established, the storage information of the data is quickly obtained, the data cleaning task is generated according to the database index information and the personalized cleaning strategy, and then the data cleaning task is executed through multiple processes to achieve regular data cleaning of the data storage platform. Therefore, intelligentization, convenience and high efficiency of rail transit multi-professional data cleaning are realized.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a database data cleaning method, device, electronic equipment and computer program product. Background technique [0002] Rail transit is a complex system that integrates multiple disciplines and types of work. It usually consists of rail routes, stations, vehicles, maintenance bases, power supply and transformation systems, communication signal systems, and command and control centers. Each profession is an independent system. In rail transit management, it is necessary to collect a variety of professional signal data for relevant systems to analyze and store, so as to achieve the purpose of rail transit operation. [0003] The data storage is mainly based on the mysql database. Through reasonable table design, the collected data will be continuously stored in the corresponding table after being parsed by the analysis service. [0004] The data stored in the database is...

AI Technical Summary

Problems solved by technology

[0006] 1. The amount of data is huge, taking up a lot of storage space, and the data cleaning is slow, which is not conducive to long-term storage;

[0007] 2. The data storage location is inconsistent, there is cross-server storage, manual cleaning is difficult, and there are security risks in operation;

[0008] 3. Data cleaning requires many dimensions, and it is difficult to achieve general coverage and support personalized expansion

Images