Unified dynamic metadata processing method for big data heterogeneous model

Abstract

The invention discloses a unified dynamic metadata processing method for a big data heterogeneous model. The method comprises the following steps: submitting the operation task to a unified dynamic metadata processing system; calling a data form analysis module to intercept, analyze and position the data of the original operation task; for the intercepted separated static data, enabling the unified dynamic metadata processing system to scan the AST so as to analyze core data related to the operation; for the intercepted separated dynamic data, analyzing and obtaining the dynamic runtime data model instances by calling a runtime reflection mechanism module; meanwhile, the REPL-based interactive parser and the runtime reflection mechanism module work cooperatively, and preparation is made for further utilizing the mechanism to carry out calculation operation reconstruction through the combined action of the reflection mechanism modules. The method has the advantages of being good in universality and high in adaptability, and the problem that efficient and convenient interoperation cannot be achieved between heterogeneous models in the prior art is effectively solved.

Description

technical field [0001] The invention relates to the technical field of computer software, in particular to a unified dynamic metadata processing method for big data heterogeneous models. Background technique [0002] The development of big data computing technology is advancing by leaps and bounds, and related computing, storage, and visualization models emerge in endlessly and evolve continuously, covering various data forms such as structured data, semi-structured data, and unstructured data, as well as batch processing and stream processing. Although the diversified technological development has brought more means and possibilities for the innovation of business applications, it has also formed obstacles for the popularization of big data technology, which is mainly reflected in the following aspects: [0003] The complexity of programming and development technology brought about by different (heterogeneous) big data computing and storage models makes users have a higher ...

AI Technical Summary

Problems solved by technology

[0003] The complexity of programming and development technology brought about by different (heterogeneous) big data computing and storage models makes users have a higher threshold in the process of learning and using;

[0004] Applications developed based on different computing and storage models are basically impossible to communicate with each other, resulting in frequent migration and replication of business data between systems, reducing the efficiency of applications;

[0005] The tight coupling between the application and the underlying computing and storage environment makes the application unable to be generalized, which brings a lot of inconvenience to the maintenance and promotion of the application;

[0006] The above problems are caused by the inability to efficiently and conveniently interoperate between heterogeneous models

Images