Function type asynchronous data distribution system and method based on coroutine

Abstract

The invention discloses a coroutine-based functional asynchronous data distribution system and method, and the method comprises the steps: designing a flow chart containing data processing logic at a front end by a user, and transmitting the flow chart to a rear-end flow chart analysis engine; the flow chart analysis engine analyzes the data representing the flow chart, generates a calculation component object and a data flow object, and correspondingly binds the calculation component object and the data flow object; the data stream object comprises a coroutine channel object used for caching data, after the calculation component executes the processing logic, the data write-out logic is executed, the data is written into the channel of the data stream object, the read-in function of the subsequent calculation component object completes the read-in of the data, and the subsequent data processing logic is carried out. According to the method, the data actually processed by the process is transmitted between the computing component object and the data stream object, both the computing component object and the data stream object run on a single server in a program form, network transmission across nodes is not involved, and the method has the advantages of high reliability, full resource utilization and high expandability.

Description

technical field [0001] The invention relates to the multi-source heterogeneous data integration field of information processing, in particular to a functional asynchronous data distribution system and method based on coroutines. Background technique [0002] With the development of information technology, all walks of life are accelerating informatization construction. However, various industries and departments build information systems according to their own needs, resulting in information collaboration among multiple departments becoming a difficult problem. At present, there are many solutions for the integration and fusion of these multi-source heterogeneous data. In addition to copying the multi-party data to a central system for processing, the common solution is to divide the integration and fusion of data into multiple steps, each step by a separate The server completes the processing, and data transmission is performed between multiple servers. For example, in Pa...

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to address the deficiencies of the existing technology, and propose a functional asynchronous data distribution system and method based on coroutines, which solves the problems of insufficient reliability, waste of resources, and poor scalability of the data distribution link in the previous data integration and fusion process. shortcoming

Images