Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Task scheduling method of Ketle cluster server

A cluster server and task scheduling technology, which is applied in the task scheduling field of Kettle cluster servers, can solve problems such as lowering Kettle system and system efficiency, unbalanced server cluster load, and increased staff task load, so as to improve customer experience, The effect of improving efficiency and improving efficiency

Active Publication Date: 2020-03-17
CHINA POWER IND INTERNET CO LTD
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] 1. The operator does not always know which server has more free resources. Whenever a new task comes, most of them can only be allocated based on feeling and experience, which can easily lead to unbalanced server cluster load
Idle ones are not being used, while busy ones are busier and the system runs slowly
[0005] 2. Manual designation increases the workload of the staff. While writing the logic program for data conversion, it is also necessary to consider the assignment of tasks, which reduces the efficiency of the Kettle system and the team using the system to complete the work.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task scheduling method of Ketle cluster server
  • Task scheduling method of Ketle cluster server
  • Task scheduling method of Ketle cluster server

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0057] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relationship between the components in a certain posture (as shown in the accompanying drawings). Relative positional relationship, movement conditions, etc., if the specific posture changes, the directional indication will also change accordingly.

[0058] In addition, in t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a task scheduling method of a Ketle cluster server, which comprises the following steps: performing coordination filtering based on the completion time of each task on each server to obtain a recommendation score of each server for each task; obtaining a load condition score of each server based on the state index of each server; based on the recommendation score of each server for each task and the load condition score of each server, obtaining a heuristic factor of each task transferred between any two servers; and performing iterative search in the cluster server based on the heuristic factor of each task transferred between any two servers in combination with an ant colony algorithm to obtain a final task scheduling allocation scheme. the data processing tasks on the Ketle system are scheduled and distributed by utilizing an artificial ant colony algorithm; the resource utilization rate of the Kettle server cluster system can be maximized, the load balance of the servers is achieved, the system performance is improved, the efficiency of the data cleaning, extracting and loading process is improved, then the customer experience is improved, the working efficiency is improved, and the benefits of enterprises are improved.

Description

technical field [0001] The invention relates to the technical field of cluster scheduling, in particular to a method for task scheduling of a Kettle cluster server. Background technique [0002] ETL (Extraction-Transformation-Loading) is a process in which one or more data sources are input, specific processing is performed, and then output to one or more data sources. The scheduling of the ETL cluster refers to the process in which the central scheduling node assigns ETL tasks to the service nodes in the cluster. Kettle, as the open-source ETL (Extraction-Transformation-Loading) tool with the largest number of users, includes two types of tasks: job and conversion, and the conversion includes specific operation components. Its powerful and concise functions are welcomed by the majority of ETL practitioners. On the Kettle platform built on cluster servers, it is often necessary to perform high-load big data extraction, conversion, cleaning, etc., and the number of service ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/50G06N3/00
CPCG06F9/4881G06F9/5083G06N3/006
Inventor 王子新傅苗胡鹏匡岳锋曹林贺群雄
Owner CHINA POWER IND INTERNET CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com