Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

A source code compiling method and system based on a MapReduce framework distributed computing environment

A mapreduce framework and distributed computing technology, applied in the field of software source code compilation, can solve the problems of simple task allocation, difficult to run efficiently and stably, and inability to restart tasks, and achieve the effect of reducing network transmission resources

Active Publication Date: 2019-05-24
CHINA STANDARD SOFTWARE
View PDF3 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] At present, the existing compilation technology has the following shortcomings: the compiler can only use multiple CPUs and multiple cores on a single device, and cannot reasonably and efficiently allocate tasks to multiple machines for compilation; the existing distributed compilation also has the following shortcomings: Lack of multi-level pipeline processing leads to low throughput of compilation tasks; task allocation is simple without multi-stage division; dynamic scheduling algorithms are complex and difficult to run efficiently and stably; resource constraints on one node lead to task failures, tasks cannot be restarted, and compilation tasks often end in failure; Network resource utilization is not planned, etc.
Therefore, there are problems with the stability, usability, and efficiency of the existing distributed compilation system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A source code compiling method and system based on a MapReduce framework distributed computing environment
  • A source code compiling method and system based on a MapReduce framework distributed computing environment
  • A source code compiling method and system based on a MapReduce framework distributed computing environment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] In order to have a further understanding of the technical solution and beneficial effects of the present invention, the technical solution of the present invention and its beneficial effects will be described in detail below in conjunction with the accompanying drawings.

[0039] Aiming at the problems of existing compiler's stand-alone processing speed and the insufficient speed, stability and expansibility of distributed compiling system, the source code compiling method and system based on MapReduce framework distributed computing environment provided by the present invention are based on staged Compile to realize the idea of ​​multi-level pipeline processing, simplify and decompose complex distributed processing tasks into task allocation algorithms and restart tasks in different stages, thereby improving system throughput and greatly improving stability.

[0040] The present invention is based on the MapReduce programming model. Map (map) means: divide calculation ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a source code compiling method and system based on a MapReduce framework distributed computing environment, and the method comprises the steps: enabling source code compilingto be better allocated to a cluster machine for compiling by analyzing the dependency relationship between files at different stages; Providing a concurrent write lock module to enable the cluster machines to write files in a mutual concurrent manner; according to the source code compiling method and system based on the MapReduce framework distributed computing environment, a MapReduce mode is used for organizing compiling work of different stages and forming an assembly line, multi-machine simultaneous compiling and multi-order concurrent assembly line is achieved, and network transmission resources are reduced; Moreover, through a synergy mechanism provided by the concurrent write lock. The source code compiling speed can be increased, and the compiling task speed and the task throughputcan be increased.

Description

technical field [0001] The present invention relates to a software source code compiling method, more specifically to a source code compiling method and system based on a MapReduce framework distributed computing environment. Background technique [0002] Under the background of informatization construction, there are more and more fields of computer software application. With the continuous deepening of application fields, computer software is constantly expanding, and the scale of software code is also increasing. Now the number of kernel source codes of the Linux operating system has reached the million level. It takes nearly 1-4 hours to compile on a server with 4 CPUs. A complete OS compilation takes about 1-2 weeks. Therefore, there is a great need for an efficient, stable, and scalable compilation technology to improve the compilation speed. [0003] Under the existing compilation technology conditions, in order to improve the compilation speed, the compilation equ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F8/41
Inventor 劳生刘艳彬吴春光刘晓东申利飞
Owner CHINA STANDARD SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products