Task orchestration method, orchestrator, equipment and readable storage medium
A task and processor technology, applied in the field of big data, can solve the problems of low task scheduling efficiency, workflow errors, workflow editing and testing taking a lot of time and resources, and achieve the effect of improving task scheduling efficiency.
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0030] figure 1 It is a flow chart of the task scheduling method provided by Embodiment 1 of the present invention. The embodiment of the present invention is aimed at manually defining the workflow by technicians in the prior art. If some tasks in the middle change, the technicians need to manually re-edit or adjust the workflow, which will inevitably lead to the existence of workflow due to manual operation errors. Errors require re-testing of the new workflow, editing and testing of the workflow consumes a lot of time and resources, and the problem of low efficiency of task arrangement provides a task arrangement method.
[0031] The method in this embodiment is applied to the orchestrator, which can be installed on the server in the big data platform. In other embodiments, the method can also be applied to other devices. This embodiment uses the orchestrator as an example to illustrate sexual description.
[0032] Such as figure 1 As shown, the specific steps of the met...
Embodiment 2
[0049] figure 2 It is a flow chart of the task arrangement method provided by Embodiment 2 of the present invention. On the basis of the first embodiment above, in this embodiment, according to the task script, the data dependency between the source table and the target table in the task is automatically generated, including: extracting the input as the corresponding task from the syntax tree of the task script The data partition of the source table of the data, and the data partition of the target table as the output data of the corresponding task, the data partition of the target table depends on the data partition of the source table; according to the syntax tree, the data partition of the source table based on the intermediate variable is generated and Data dependencies between data partitions of the target table. Such as figure 2 As shown, the specific steps of the method are as follows:
[0050] Step S201, acquiring a task script, which is a descriptive language scr...
Embodiment 3
[0123] Figure 5 It is a schematic structural diagram of the arranger provided by Embodiment 3 of the present invention. The orchestrator provided in the embodiment of the present invention can execute the processing flow provided in the embodiment of the task orchestration method. Such as Figure 5 As shown, the orchestrator 30 includes: a task script acquisition module 301 , a dependency generation module 302 and a task driver module 303 .
[0124] Specifically, the task script obtaining module 301 is used to obtain the task script, and the task script is a descriptive language script.
[0125] The dependency generating module 302 is configured to automatically generate the data dependency between the source table and the target table in the task corresponding to the task script according to the task script.
[0126] The task driving module 303 is used to drive the execution of tasks in a data-driven manner according to the data dependency relationship between the source ...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com