Data processing method based on Oracle

Abstract

The invention discloses a data processing method based on Oracle. The method comprises the following steps that a) the storage mode, the migration mode and the cleaning mode of data in each table are respectively set by aiming at different data table entities in an Oracle database; b) at least one set table data cleaning mode in the Oracle database is combined for forming data cleaning operation; c) a concrete executing time window is allocated by aiming at the data cleaning operation, and in addition, the dispatching operation execution is carried out in the preset time window; d) according to an automatic operation dispatching execution result, monitoring information of data migration and cleaning is summarized. The data processing method has the advantages that the storage mode, the migration mode and the cleaning mode are respectively set for different data table entities, and the dispatching data cleaning operation realizes the periodization and automation of the data migration and cleaning process, so the data operation workload is reduced, in addition, the zero machine halt time and the zero influence on the application in the migration and cleaning process are realized.

Description

technical field [0001] The invention relates to a data processing method, in particular to an Oracle-based data processing method. Background technique [0002] Since most IT systems are production systems that operate 7*24 hours a day, a large amount of data is continuously generated during the production operation. These generated data come from user additions, orders, work orders, user account information, etc. Usually, these data are continuously generated by system applications, but the system applications themselves usually do not effectively process the generated data. If the historical data cannot be effectively cleaned, the data can only enter and cannot enter; at the same time, although sufficient redundant capacity is planned during the system launch planning, due to the continuous increase and launch of new services in the system, the data grows The speed is still far faster than planned. [0003] The rapid growth of data often brings all kinds of problems and...

AI Technical Summary

Problems solved by technology

[0005] 2. Impact on system performance: Although data access is generally concentrated on recent data, the database still needs to scan the data or indexes of the entire table. The data volume of a single table is huge, resulting in a significant reduction in the efficiency of overall data query

[0006] 3. Maintenance and security: The large amount of table data makes common database maintenance difficult, including index creation, data migration and data maintenance, etc.

At the same time, due to the characteristics of the IT system of the telecom operator, its core business database continuously generates a large amount of data, and the storage expansion speed is far from meeting its growth demand. It is necessary to perform data life cycle management and migration from time to time to ensure its normal work.

[0014] However, the shortcomings of the current application technology solution lie in the following problems: 1. The data migration efficiency is too low, and the migration time is too long; 2. The process of data cleaning has an impact on the business system, and it needs to be implemented during the shutdown maintenance window

For example, truncate partition or drop partition is generally used to clean up data in the partition mode. At this time, if the table has a global index or primary key, the index needs to be automatically maintained or manually rebuilt, which will affect the running system application

3. High dependence on people and low degree of automation

Images