Method and device of processing repeated data

Abstract

The embodiment of the invention provides a method and a device of processing repeated data. The method comprises the steps: receiving a reading request used for reading data in a physical block, wherein the reading request comprises the information of a mapping table which corresponds to the physical block; writing the data in the physical block into a memory according to the information of the mapping table; and deleting repeated data written into the physical block in the memory. When the reading request is received, the thread of deleting the repeated data is touched by the reading requestfor changing the thread of deleting the repeated data into a passive mode; the operation of deleting the repeated data has less influence on reading and writing the data; and moreover, extra I / O expense is not needed. The data in the physical block are directly written into the memory without the need of CACHE processing, thereby reducing the influence of the operation of deleting the repeated data on the reading and writing performance of an SSD main memory array.

Description

technical field [0001] The invention relates to data processing technology, in particular to a repeated data processing method and device. Background technique [0002] A solid-state hard drive (Solid-state hard drive, hereinafter referred to as SSD) has been applied to a primary storage array because of its high performance. However, due to the high price of SSD storage media, it is necessary to make full use of the storage space in the SSD during use, and the data stored in the SSD may have data duplication. Duplicate data will occupy the storage space in the SSD, so you need to pass Data deduplication technology deletes duplicate stored data in SSD. [0003] In the prior art, there are many methods for data deduplication, for example: Method 1, the synchronous method, also called the in-band method, that is, when the data in the memory is written into the SSD, the data first resides in the memory , and then call the deduplication thread to identify the data to be writte...

AI Technical Summary

Problems solved by technology

[0004] In the process of implementing the present invention, the inventors found the following problems in the prior art: in method one, the data can only be written into the SSD after the deduplication operation is completed, and the data to be written resides in the memory, which cannot be effectively Therefore, the data deduplication operation has a greater impact on the read and write performance of the SSD primary storage array.

In the second method, since a separate thread needs to be started to perform the data deduplication operation, additional input / output (Input / Output, hereinafter referred to as: I / O) overhead is added, which causes the system bandwidth to be occupied, and also makes the deduplication operation harmful. The read and write performance of the SSD primary storage array is greatly affected

Images