A data exchange method, device, equipment, medium

Abstract

The present application discloses a data exchange method, device, equipment, and medium. The method includes: obtaining back-end database information in a distributed database; and then controlling the number of ETL working nodes created and the number of back-end databases according to the back-end database information Equal data exchange task threads; and control the ETL work node to execute the data exchange task threads to directly extract target data concurrently from the source database, and directly concurrently load the target data to the target database. It can be seen that, after the application obtains the back-end database information in the distributed database, it will control the ETL working nodes to create data exchange task threads whose number is equal to the number of back-end databases, and execute a plurality of said data exchange task threads, so as to Direct concurrent reading and writing of the database, the data exchange task thread does not go through the middleware, no longer has the single-point performance limitation of the middleware, realizes the concurrent reading and writing of data, and improves the data exchange performance of the distributed database.

Description

technical field [0001] The present application relates to the technical field of databases, in particular to a data exchange method, device, equipment, and medium. Background technique [0002] ETL (Extract-Transform-Load) is used to describe the process of extracting, transforming, and loading data from the source to the destination. When using ETL tools to process data exchange tasks in distributed databases, it is necessary to use middleware to parse SQL semantics and perform SQL business proxying. Although the overall proxy capability of the middleware can be improved through the horizontal expansion capability of the middleware, the data exchange of a specific data table can only be performed by one middleware agent, which has a single-point performance bottleneck and low data exchange performance. Contents of the invention [0003] In view of this, the purpose of this application is to provide a data exchange method, device, equipment, and medium, so that the perfor...

AI Technical Summary

Problems solved by technology

Although the overall proxy capability of the middleware can be improved through the horizontal expansion capability of the middleware, the data exchange of a specific data table can only be performed by one middleware agent, which has a single-point performance bottleneck and low data exchange performance.

Images