Multi-core processor architecture based on MapReduce programming model

Abstract

The invention discloses a multi-core processor architecture based on a MapReduce programming model, is used for quickening speed for a computational node to process a heavy data stream in a computer network and simplifying the parallel programming complexity of the computational node. In the processor, input data is output after being successively processed by a Split segmentation scheduling module, a MapReduce block assembly line module and a Merge module. The modules are connected through a multilayer on-chip interconnection structure, the Split segmentation scheduling module segments the input data into small blocks, and the MapReduce block assembly line module is controlled to process the small blocks; each block assembly line in the MapReduce block assembly line module consists of a Map processor, a Reduce processor and two Ping-Pong-type data caches, wherein the Map processor and the Reduce processor carry out MapReduce processing on data in the data caches in a Ping-Pong form; and finally, the Merge module adopts a hierarchical structure to merge and sort the results of the MapReduce, and obtains a final result.

Description

technical field [0001] The invention belongs to the technical field of multi-core processor acceleration, in particular to a multi-core processor architecture based on a MapReduce programming model. Background technique [0002] In recent years, the Internet of Things technology has developed rapidly. It combines various sensors and intelligent processing, and uses various intelligent technologies such as cloud computing and pattern recognition to analyze, process and process meaningful data from the massive information obtained by sensors. To meet the needs of different applications such as business, security, and urban management. With the gradual expansion and deepening of the application field of the Internet of Things, the number of various sensors is increasing geometrically, and the amount of collected information is bound to increase explosively, followed by the processing of massive big data. At present, the Internet of Things mostly adopts a centralized storage an...

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved by the present invention is to solve the problem of slow processing speed of general processors when performing large data calculations, and proposes a multi-core processor architecture based on the MapReduce programming model, which is used to accelerate the speed of computing nodes to process large data streams. And simplify the complexity of computing node parallel programming

Images