Data decompression device and method

Abstract

The invention relates to a data decompression device, which comprises a driving module, a hardware scheduling module and multiple on-chip decompression cores, wherein the driving module is used for combining a plurality of concurrent decompression requests received from a plurality of application program processes into a decompression command, transmitting the decompression command to the hardware scheduling module and returning a decompression result to the application program processes; the hardware scheduling module is used for analyzing the received decompression command, distributing independent decompression cores to a plurality of decompression requests in the decompression command for processing and transmitting interruption and state information to a driving program after the decompression requests are all processed; and the multiple on-chip decompression cores comprise a plurality of independent decompression cores for executing decompression processing. The invention also relates to a data decompression method. In the invention, by performing concurrent processing on a plurality of decompression requests of a plurality of data decompression threads through the multiple on-chip decompression cores, the CPU (Central Processing Unit) resource is not required to be consumed, the decompression bandwidth can also be increased, and the decompression speed is increased.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a processing device and processing method for decompressing data. Background technique [0002] In the large-scale data processing of the Internet, data compression and decompression are very important and commonly used means. It can greatly increase the effective capacity of the disk, improve the effective bandwidth of I / O read and write and network transmission, and thus effectively reduce the The cost of the Internet Data Center (Internet Data Center, referred to as IDC) and the execution speed of the application layer program are improved. [0003] In traditional solutions, data compression and decompression generally use Gzip (a file compression program of GNU free software) compression algorithm. There are a large number of serial variable-length bit operations in this Gzip algorithm. Since the CPU processes data and accesses memory in units of fixed-length bits, the CPU nee...

AI Technical Summary

Problems solved by technology

There are a large number of serial variable-length bit operations in this Gzip algorithm. Since the CPU processes data and accesses memory in units of fixed-length bits, the CPU needs to process data and access memory from regular data. Take out the really effective data, thus adding additional operations, resulting in low software processing efficiency, for example, the CPU with a main frequency of 2.66GHz compresses the bandwidth of 50MB / s, and decompresses the bandwidth of 200MB / s

However, the decompression operation of the Gzip compression algorithm currently used is implemented in software. This method has a low degree of parallelism. In large-scale data processing, the data stream to be compressed and decompressed is huge, and the CPU is used for compression and decompression. Compression processing will inevitably consume a large amount of CPU resources, which will increase the load on the machine

Images