Big data hybrid scheduling model on private cloud condition

Abstract

The invention relates to the technical field of cloud computing and big data, in particular to a big data hybrid scheduling model on the private cloud condition. The big data hybrid scheduling model comprises an access control module, a task decomposition module, an overall situation scheduler module based on virtual resources, a resource virtualization mapping module, a virtual resource monitoring module, a physical resource monitoring module, and a local scheduler module based on physical resources. The big data hybrid scheduling model is a hybrid scheduler including physical resource scheduling, virtual resource scheduling, overall situation scheduling and local scheduling. User tasks are decomposed into a plurality of independent subtasks which are distributed on different clusters or computational nodes. Parallel and rapid execution of the tasks is ensured, network and data transfer costs are reduced, physical resource scheduling is utilized for further decomposing subtasks, cluster resources are utilized to be maximum extent, and the task execution time is further shortened.

Description

technical field [0001] The invention relates to the technical fields of cloud computing and big data, in particular to a big data hybrid scheduling model under the condition of private cloud. Background technique [0002] With the rapid growth of data in the Internet, mobile Internet, Internet of Things, finance and other fields, using big data technology to process huge data information can achieve huge economic benefits and social value. Big data is not only a surge in data volume, but also an increase in data complexity, but it also requires large-scale storage and real-time processing of data. [0003] Cloud computing can manage and schedule a large number of network-connected resources in a unified manner, forming a resource pool to provide on-demand services to users, and realize the effective storage and processing of big data. Cloud computing includes public cloud, private cloud and hybrid cloud. Among them, the private cloud is built for a company or organization t...

AI Technical Summary

Problems solved by technology

The main challenges faced by the big data scheduling under the private cloud are: 1) The scale of the private cloud is relatively small, the resources are relatively limited when processing big data, and real-time performance is difficult to guarantee; 2) Data migration will occur during the processing of big data, further increase resource consumption

[0005] After investigation, no public patents on the big data hybrid scheduling model under private cloud conditions have been found

Images