Data source general processing framework construction method and device and data source processing method and device

Abstract

Embodiments of the invention provide a data source general processing framework construction method and device and data source processing method and device. According to the data source general processing framework construction method and device, a general processing framework which comprises a uniform operation interface, a general abstract class and a specific abstract class can be constructed,wherein the uniform operation interface is used for unifying and classifying operation interfaces of various types of data sources; and the general abstract class and the specific abstract class are used for hierarchically packaging common data operations and specific data operations of the data sources. By applying the constructed general processing framework, data operations for various types ofdata sources can be completed through the general abstract class and the specific abstract class in the general processing framework by calling the uniform operation interface, and interface connection does not need to be established for each database, so that the connection number of data source system resources is decreased, the data source operation logic is simplified, the data source systemresource occupation is reduced, and the data source system operation speed and efficiency are improved.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular to a general processing framework construction method, processing method and device for heterogeneous data sources. Background technique [0002] With the development of computer technology and Internet technology and the advent of the era of big data, the amount of data on the Internet has grown rapidly. In the Internet world, various types of data are continuously generated, disseminated and accumulated. Therefore, in the prior art, in order to realize the processing, analysis and management of a large amount of data, many data storage technologies have emerged. These data storage technologies store a certain type of data in a specific format to generate a data source. [0003] Data sources in the prior art mainly include relational data source (SQL) and non-relational data source (NoSQL), wherein: relational data source (SQL) includes MySQL database, Oracle ...

AI Technical Summary

Problems solved by technology

However, the JDBC interface program in the prior art can only be used to process non-relational data sources, and the JDBC interface program needs to connect to the data source every time it processes the data source, and needs to close the connection after the data source is processed. Therefore, if the data source is frequently processed, the JDBC interface program needs to frequently create and close connection data, wasting system resources, thereby reducing the data source processing efficiency

Although the Spring Boot framework in the prior art can handle some relational data sources (SQL) and non-relational data sources (NoSQL), the types of applicable data sources are limited and not universal; Sources use different processing methods. When there are many types of data sources, the Spring Boot framework becomes bloated and complicated, thereby reducing the efficiency of data source processing.

Images