Data processing method and device, computer equipment and storage medium

Abstract

The invention provides a data processing method and device, computer equipment and a storage medium. The method comprises the following steps: determining a target plug-in matched with a data processing demand of a target service system in at least one plug-in; generating a job for performing data processing on the target service system in response to an editing operation on the job attribute in the configuration page of the target plug-in; generating a job flow of the target service system according to at least one job used for performing data processing on the target service system; under the condition that the user sending the data processing request has the processing authority to the target service system, the target operation flow of the target service system indicated by the data processing request is called to process the data of the target service system pulled in the database cluster. According to the method, data processing of various data processing scenes is realized through a unified data development system, so that the development difficulty can be reduced, the development efficiency can be improved, and the online data processing capability can be improved on the basis of simplifying the data processing operation.

Description

technical field [0001] The present invention relates to the field of computer technology, and more specifically, to a data processing method, device, computer equipment and storage medium. Background technique [0002] The traditional big data development method needs to query all kinds of data from the database of the business system, and then process the data from different data sources on different platforms. Moreover, developers of each platform need to use separate databases, such as relational MySQL, Oracle, etc., and non-relational Mongo, Redis, etc. [0003] The data of the platform is stored on one or more machines, and may not be fully stored. Developers need to process and filter valuable data through data analysis in a data collection with multiple sources, formats, and large volume The corresponding platform performs data processing, and the problems caused by such processes and models are obvious. [0004] First of all, there is no unified data development sy...

AI Technical Summary

Problems solved by technology

[0003] The data of the platform is stored on one or more machines, and may not be fully stored. Developers need to process and filter valuable data through data analysis in a data collection with multiple sources, formats, and large volume The corresponding platform performs data processing, and the problems caused by such processes and models are obvious

[0004] First of all, there is no unified data development system for massive data, which undoubtedly greatly increases the difficulty of platform development and reduces development efficiency

Moreover, when developers need to use multiple platforms and multiple types of databases for one data processing, they need to perform different configurations, which is cumbersome and time-consuming

In addition, due to the limitation of the number of machines used by the platform, a lot of data cannot be completely stored and flowed, which leads to the phenomenon of "lack of data", so the online processing capability of the platform is also extremely low

Images