Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A high-performance mapreduce implementation method based on virtual machine dynamic migration

An implementation method and dynamic migration technology, which is applied in the direction of program start/switch, software simulation/interpretation/simulation, multi-program device, etc., can solve problems such as large network bandwidth occupation and unreasonable utilization of server resources, etc., to improve security performance, increase overall utilization, and improve performance

Active Publication Date: 2017-10-20
长沙麓云信息科技有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0019] The present invention proposes a high-performance MapReduce implementation method based on virtual machine dynamic migration, and its purpose is to overcome the problems of large network bandwidth occupation and unreasonable utilization of server resources in the above-mentioned prior art MapReduce implementation method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A high-performance mapreduce implementation method based on virtual machine dynamic migration
  • A high-performance mapreduce implementation method based on virtual machine dynamic migration
  • A high-performance mapreduce implementation method based on virtual machine dynamic migration

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] A high-performance MapReduce implementation method based on virtual machine dynamic migration, comprising the following steps:

[0037] Step 1: Build a distributed running Hadoop system with virtual machines on the physical server;

[0038] Step 2: The user Client submits the job to the Hadoop system described in step 1, and the Hadoop system decomposes the job into split data blocks;

[0039] Step 3: Extract sample elements from the split data block according to the rules, and then use all the extracted sample elements as a simulation sample group;

[0040] Step 4: Process the simulated sample group by the Map task to obtain the key-value pair corresponding to each split data block, and partition the data according to the key value to generate a partition;

[0041] Step 5: Each partition is processed by a Reduce task, and the Map task source ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a high-performance MapReduce implementation method based on dynamic migration of virtual machines. The virtual machine cloud management platform of CloudStack is used to monitor the cluster state in real time, to sample split data blocks, and to count the data received on the virtual machine where the Reduce task is located. For the Map task source corresponding to each key value, the Map task source that generates the most key values ​​is obtained; the physical machine of the virtual machine where the obtained Map task source is located is used as the migration target of the virtual machine where the corresponding Reduce task is located; as the core, through the virtual machine dynamic Migration enables the Reduce task to receive the output of the Map task with the same key value as locally as possible, saving network bandwidth.

Description

technical field [0001] The invention relates to a high-performance MapReduce implementation method based on virtual machine dynamic migration. Background technique [0002] Currently, cloud computing, as a business computing model, promotes the proposal of networked operating systems. The core technology of cloud computing is virtualization technology. As the basis of the networked operating system, the virtual machine cloud management platform is built on a cluster composed of a large number of servers and other hardware resources to provide users with an available operating system environment, and there is no connection between them. Impact, that is, the realization of IaaS (Infrastructure as a Service). [0003] Under the existing technology and operation mode, in the virtual machine cloud management platform in the cloud computing system, the virtual machine is created for the user by providing the template image of the operating system. Disk images of cloud systems. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/455
Inventor 唐卓江林刚曾优杨黎马稳祁玲
Owner 长沙麓云信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com