Multiple data stream processing method based on MIC co-processor

Abstract

The invention discloses a multiple data stream processing method based on an MIC co-processor. The method comprises the steps that a central processing unit (CPU) acquires a data source and transmits multiple data streams which arrive in parallel to the MIC co-processor through a PCIE bus; the MIC co-processor performs parallel calculation on the data streams by adopting a four-layer sliding window model and carries out a multiple data stream query algorithm to obtain a data stream query result; the MIC co-processor returns the data stream query result to the CPU; the CPU obtains an IO output task of a user according to the data stream query result and carries out the IO output task. According to the multiple data stream processing method based on the MIC co-processor, the parallel performance and real-time performance of multiple data stream processing are improved.

Description

technical field [0001] The invention relates to the technical field of big data high-performance computing, in particular to a multi-data stream processing method based on an MIC coprocessor. Background technique [0002] Real life, high-speed network fault diagnosis, transaction data flow in retail business, online auction, transaction log, Web tracking and personalization, medical monitoring, phone record data flow in communication field, packet flow in network monitoring, environment The temperature monitoring data and the image data stream sent back by the satellite form a data form different from the static data in the traditional database. The arrival of data in data streams is fast, time-varying, unpredictable and infinite in the form of data streams, and it is impossible to completely store the original data. Moreover, the amount of data generated by these data streams is growing rapidly in many application fields, and the applications that generate data streams usu...

AI Technical Summary

Problems solved by technology

When the data scale is large, the data is often used as the medium of disk or tape, so the execution of query operations requires a large number of I / O exchanges, which is inefficient and cannot meet the needs of real-time systems.

Due to the ever-changing and unpredictable characteristics of multi-data streams and the generation of multi-data stream bursts, higher requirements are placed on the load capacity of multi-data streams. At the same time, multi-data stream processing is difficult to meet real-time requirements due to excessive time overhead. , so the parallelism and real-time performance of multi-data stream processing are very low

Images