Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Cross-cluster task scheduling system and method

A task scheduling and cross-cluster technology, applied in the computer field, can solve problems such as inconsistent location, inconvenient scheduling, supervision and management, and slow task processing speed

Active Publication Date: 2020-01-21
恩亿科(北京)数据科技有限公司
View PDF8 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There is an association between tasks and tasks between multiple clusters. For example, a submitted task in a computing engine cluster may need to combine data in a distributed database cluster and a distributed system infrastructure cluster (Hadoop Distributed FileSystem, hadoop cluster) To perform calculations, in the prior art, in order to combine the data between the two clusters, the computing engine cluster first stores and calculates the data in the distributed database cluster and the data in the Hadoop cluster, and then stores and calculates the calculation results of the two clusters , when the workflow scheduler schedules tasks for the computing engine cluster, due to the inconsistency of data in the cluster, it is not convenient for task scheduling, supervision and management. As a result, when the workflow scheduler schedules tasks, the computing engine cluster cannot The problem of slow task processing caused by providing the data required by the task

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-cluster task scheduling system and method
  • Cross-cluster task scheduling system and method
  • Cross-cluster task scheduling system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the technical solutions in the embodiments of the present application The drawings are only for the purpose of illustration and description, and are not used to limit the protection scope of the present application. Additionally, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented in accordance with some embodiments of the application. It should be understood that the operations of the flowcharts may be performed out of order, and steps that do not have a logical context may be performed in reverse order or simultaneously. In additio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a cross-cluster task scheduling system and method. The task scheduling system comprises a workflow scheduler, a distributed database cluster, a distributed system infrastructure cluster and a computing engine cluster. According to the invention, the workflow scheduler issues the instruction for processing the target task to the computing engine cluster, and data required for obtaining the target task is issued to the distributed database cluster. Through a command line tool, the distributed database cluster writes data required by the target task into a distributed system infrastructure cluster. The interaction between the distributed database cluster and the distributed system infrastructure cluster is broken through, so that the calculation engine cluster can obtain all data required for calculating the target task only by reading the distributed system infrastructure cluster, then the task result of the target task is calculated, and the task processing process is accelerated.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a cross-cluster task scheduling system and method. Background technique [0002] With the growth of data processing scale, the traditional stand-alone computing model can no longer meet the growing demand for information services. In order to improve the stability of the system and the data processing and service capabilities of the network center, cluster technology is usually used. A cluster is a group of independent computers interconnected through a high-speed network. They form a computer group and are managed in a unified manner. Clusters can achieve high computing speed, complete calculations with a large amount of computing, have high responsiveness, and can reduce overall operation and maintenance costs, so they have been used more and more. [0003] In the same cluster, tasks can be scheduled according to the order of the directed acyclic graph (DirectedAc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48
CPCG06F9/4843
Inventor 余婷婷
Owner 恩亿科(北京)数据科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com