Data storage method and device

Abstract

The embodiment of the invention discloses a data storage method and a data storage device, which are applied to the technical field of data storage. The data storage method comprises the following steps: acquiring semantic information contained in a file to be stored by analyzing the file to be stored, wherein the semantic information is used for indicating the attribute information of the file to be stored; decomposing the file to be stored into at least two semantic data blocks according to the semantic information and acquiring content identifiers of the semantic data blocks, wherein the content identifiers are used for indicating the content information of the semantic data blocks; storing the at least two semantic data blocks and a first mapping relationship between the information of the file to be stored and the content identifiers, wherein the information of the file to be stored is used for identifying the positions of the semantic data blocks; and storing the semantic data blocks and a second mapping relationship between the content identifiers and the storage positions of the semantic data blocks. Therefore, the file searching efficiency is improved, and the data management and retrieval are convenient.

Description

technical field [0001] The invention relates to the field of data storage, in particular to a data storage method and equipment. Background technique [0002] In today's data storage field, the increase in data volume leads to an increase in the total cost of ownership (Total Cost of Ownership, TCO) of the storage system including hardware costs, maintenance costs, and labor costs. Reducing the amount of data in the process of data transmission, data storage and data management has become a requirement in large-scale distributed storage systems, and data deduplication technology is a method of eliminating duplicate data through lossless data compression at the internal level of files. technology, which has good application value in storage systems of application types such as data archiving. [0003] The existing data deduplication technology is mainly: based on the bit level, the file is divided into non-overlapping data blocks, and then only one copy of the blocks with th...

AI Technical Summary

Problems solved by technology

[0005] After the files are stored according to the existing data storage method, when looking for the files, the system needs to retrieve the relevant non-repeated large blocks first. These large blocks are randomly distributed in the storage device, and the files need to be reconstructed one by one. Then check whether the content of the file matches the required query, which will reduce the efficiency of file search; in addition, because the existing technology is based on the bit-level segmentation of the file, so the process of data storage may have little difference The two files are divided into almost completely different data blocks, and duplicate data cannot be deleted very well

Images