Method and system for scheduling Zeppelin tasks, computing device and storage medium

A storage medium and task technology, applied in the field of big data processing, can solve problems such as high labor cost and low execution efficiency

Pending Publication Date: 2020-05-15
HANGZHOU YITU MEDIAL TECH CO LTD
View PDF7 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to solve the problem of high manpower cost and low execution efficiency in Zeppelin in the prior art where multiple task scheduling with dependencies needs to be manually set.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for scheduling Zeppelin tasks, computing device and storage medium
  • Method and system for scheduling Zeppelin tasks, computing device and storage medium
  • Method and system for scheduling Zeppelin tasks, computing device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] The implementation of the present invention will be illustrated by specific specific examples below, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention will be presented in conjunction with a preferred embodiment, it does not mean that the features of the invention are limited to this embodiment. On the contrary, the purpose of introducing the invention in conjunction with the embodiments is to cover other options or modifications that may be extended based on the claims of the present invention. The following description contains numerous specific details in order to provide a thorough understanding of the present invention. The invention may also be practiced without these details. Also, some specific details will be omitted from the description in order to avoid obscuring or obscuring the gist of the present invent...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for scheduling Zeppelin tasks. The method comprises the following steps that Zeppelin generates notebook containing a plurality of tasks; analyzing to obtain a dependency relationship between the tasks; generating an arrangement file according to the dependency relationship; completing tasks from orchestration files. The execution sequence of the Zeppelin tasks with the dependency relationship does not need to be manually set, and the Zeppelin tasks with the dependency relationship can be automatically scheduled. The invention further provides a system for scheduling the Zeppelin tasks, computing equipment and a storage medium.

Description

technical field [0001] The invention relates to the field of big data processing, in particular to a method, system, computing device and storage medium for scheduling Zeppelin tasks. Background technique [0002] At present, in the field of data governance, there are various ETL (data warehouse technology) tools and scheduling systems, such as Kettle, Nifi, Zeppelin, Sqoop, etc. Kettle is a commonly used ETL tool software, but it only supports stand-alone deployment and cannot meet the data processing requirements in the distributed environment of big data. Nifi is an excellent data diversion and task scheduling system, but it is somewhat helpless for the need to execute a large amount of SQL and unified management. Zeppelin is an excellent web data-driven interactive cloud development environment, which is convenient for ETL personnel and program developers to use, without worrying about environmental issues, and can focus more on the realization of business logic. [00...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/25G06F9/48
CPCG06F16/254G06F9/4881G06F2209/484
Inventor 郑永升石磊石权
Owner HANGZHOU YITU MEDIAL TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap