Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Mapreduce job scheduling method and device based on maximizing revenue for big data platform

A job scheduling and job technology, applied in data processing applications, marketing, instruments, etc., can solve the lack of solutions, does not consider the impact of job scheduling results on platform resource utilization, and does not take into account the user's accurate consideration of urgent needs, etc. problems, to shorten the completion time, achieve a win-win situation, and improve the utilization of platform resources.

Active Publication Date: 2020-06-05
SHANDONG UNIV
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Based on the original revenue model strategy and the existing job scheduling strategy, the service provider did not take into account the user's accurate consideration of the job deadline and the urgency of job execution, and did not consider the impact of job scheduling results on platform resource utilization
[0004] How to formulate an efficient job scheduling strategy in a certain big data computing resource environment so that the service provider can obtain the maximum benefit and at the same time determine the more accurate completion time of each job for the user and achieve the maximum utilization of platform resources. lack of effective solutions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mapreduce job scheduling method and device based on maximizing revenue for big data platform
  • Mapreduce job scheduling method and device based on maximizing revenue for big data platform
  • Mapreduce job scheduling method and device based on maximizing revenue for big data platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0072] This embodiment discloses a MapReduce job scheduling method under the cloud service provider reward and punishment income mode, including the following steps:

[0073] Receive jobs submitted by users, and obtain the execution time of each round of Map tasks and Reduce tasks for each job, as well as the number of tasks;

[0074] According to the Map and Reduce task execution time and the number of tasks of each job, and according to the reward and punishment revenue model, determine the maximum number of rounds combination scheme set and the maximum standard time for each job in different reward and punishment stages;

[0075] According to the reward and punishment revenue model, the job scheduling strategy is obtained based on the maximum number of rounds scheme for each job.

[0076] Job Earning Model

[0077] According to the above income and compensation information about the operation, combined with the number of resources that can be allocated in the platform, the...

Embodiment 2

[0138] The purpose of this embodiment is to provide a computing device.

[0139] A MapReduce job scheduling optimization device under the reward and punishment income mode of a cloud service provider, including a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the following steps when executing the program, include:

[0140] Receive jobs submitted by users, and obtain the execution time of each round of Map tasks and Reduce tasks for each job, as well as the number of tasks;

[0141] According to the Map and Reduce task execution time and the number of tasks of each job, and according to the reward and punishment revenue model, determine the maximum number of rounds combination scheme set and the maximum standard time for each job in different reward and punishment stages;

[0142] According to the reward and punishment revenue model, the job scheduling strategy is obtained based on the maximum numbe...

Embodiment 3

[0144] The purpose of this embodiment is to provide a computer-readable storage medium.

[0145] A computer-readable storage medium, on which a computer program is stored for calculating the similarity of fingerprints. When the program is executed by a processor, the following steps are performed:

[0146] Receive jobs submitted by users, and obtain the execution time of each round of Map tasks and Reduce tasks for each job, as well as the number of tasks;

[0147] According to the Map and Reduce task execution time and the number of tasks of each job, and according to the reward and punishment revenue model, determine the maximum number of rounds combination scheme set and the maximum standard time for each job in different reward and punishment stages;

[0148] According to the reward and punishment revenue model, the job scheduling strategy is obtained based on the maximum number of rounds scheme for each job.

[0149] The steps involved in the above embodiments 2 and 3 co...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a MapReduce job scheduling method and device applied to a cloud service provider reward and punishment profit mode. The method includes the following steps that: jobs submittedby a user are received, the execution time of each round of Map task and each round of Reduce task of each job, and the task quantities of the Map tasks and Reduce tasks of each job are obtained; themaximum round combination plan set and maximum standard time of each job in different reward and punishment stages are determined according to the execution time and task quantities of the Map tasksand Reduce tasks of each job on the basis of the reward and punishment profit mode; and job scheduling strategies are obtained on the basis of the maximum round plan of each job according to the reward and punishment profit mode. According to the method and device of the invention, the profit and payment cost of each job are evaluated according to an objective of maximizing the profits of a service provider, and therefore, the service provider can obtain maximum profits, the resources of a platform can be utilized to the greatest extent, and a job can be completed with the shortest time.

Description

technical field [0001] The invention belongs to the field of cloud platform job scheduling optimization, and in particular relates to a MapReduce job scheduling method and device under the cloud service provider reward and punishment revenue mode. Background technique [0002] In recent years, with the explosive growth of various data, the demand for more efficient analysis and processing of massive data has become more and more urgent. Traditional data processing technologies and tools can no longer meet the current analysis and processing needs, so the emerging big data computing platform provides strong support for solving new needs. Due to the contradictory relationship between the efficient processing requirements and processing costs of large amounts of data, big data computing platform service providers provide users with convenient and low-cost computing services. The service provider has established a public big data computing platform based on the existing big dat...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06Q10/06G06Q30/02
CPCG06Q10/0631G06Q10/06312G06Q30/0207
Inventor 史玉良胡静李庆忠孔兰菊闫中敏
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com