Mass file storage method and device for distributed type system

Abstract

The invention provides a mass file storage method and device for a distributed type system. The method comprises the steps of encoding 2n data files to be stored to generate m global verifying block files; dividing the 2n data files into k groups, encoding each group of data files to generate corresponding local verifying block files; storing the 2n data files, the m global verifying block files and the k local verifying block files in 2n+m+k severs in the distributed type system according to a mode that only one file is stored in one server. According to the scheme, encoding treatment is conducted on original data, the global verifying block files and the local verifying block files are calculated, then, original data files and the verifying block files simply need to be stored in the distributed type system instead of storing multiple copies of the original data files, and thus the usage rate of storage equipment is obviously increased on the premise of guaranteeing that the storage reliability is not lowered.

Description

technical field [0001] The invention relates to the field of information technology, in particular to a method and device for storing massive files in a distributed system. Background technique [0002] With the rapid development of information technology, the explosive growth of massive data has brought about data storage and security issues. In order to improve the security and availability of data storage, most traditional distributed data storage systems adopt the method of multiple copies, copying a data file into multiple copies and storing them in multiple servers in the distributed system respectively. If any server with a copy of the data file is available, the data file can be accessed to avoid data loss and inaccessibility problems caused by network failure of a single server or disk damage, thereby achieving disaster recovery through multi-copy redundancy. [0003] This multi-copy storage method improves the reliability of persistent data storage by increasing r...

AI Technical Summary

Problems solved by technology

[0003] This multi-copy storage method improves the reliability of persistent data storage by increasing redundancy. Although it has the advantages of easy data recovery and reading pressure sharing, the utilization rate of storage devices is relatively low; moreover, for those who need to store PB-level For companies with massive data, the cost of data storage operation and maintenance is relatively high

Images