Systems and methods for repair redundancy control for large erasure coded data storage

Abstract

Systems and methods which implement storage system data repair control techniques, such as may provide a feedback control structure for regulating source object redundancy and / or repair bandwidth in the storage system. Embodiments control a source object redundancy level to be used in a storage system by analyzing source objects represented in a repair queue to determine repair rate metrics for the source objects and determining a source object redundancy level based on the repair rate metrics. For example, embodiments may cause more redundant fragments for each source object to be generated and stored during repair where the repair rate metrics indicate an increase in storage node failure rate. Additionally, embodiments may determine a per storage object repair rate (e.g., a repair rate preference for each of a plurality of source objects) and select a particular repair rate (e.g., a maximum repair rate) for use by a repair policy.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims the benefit of U.S. Provisional Patent Application No. 62 / 211,303, entitled, “SYSTEMS AND METHODS FOR REPAIR RATE CONTROL FOR LARGE ERASURE CODED DATA STORAGE”, filed on Aug. 28, 2015, which is expressly incorporated by reference herein in its entiretyDESCRIPTION OF THE RELATED ART[0002]The creation, management, storage, and retrieval of electronic data has become nearly ubiquitous in the day-to-day world. Such electronic data may comprise various forms of information, such as raw data (e.g., data collected from sensors, monitoring devices, control systems, etc.), processed data (e.g., metrics or other results generated from raw data, data aggregations, filtered data, etc.), produced content (e.g., program code, documents, photographs, video, audio, etc.), and / or the like. Such data may be generated by various automated systems (e.g., network monitors, vehicle on-board computer systems, automated control systems, e...

AI Technical Summary

Benefits of technology

The patent text describes a computer-readable medium that includes instructions for controlling the repair rate of a storage system. The computer program analyzes multiple source objects in a repair queue to determine the optimal repair rate based on various metrics. By implementing this repair rate, the system can recover the source data and achieve efficient repair, while prioritizing the repair of the most important objects.

Problems solved by technology

However, the individual storage nodes are somewhat unreliable in that they can intermittently fail, in which case the data stored on them is temporarily unavailable, or permanently fail, in which case the data stored on them is permanently lost (e.g., as represented by the failure of storage node 130-2 in FIG. 1C).

This can result in inefficiencies with respect to the use of resources, such as communication bandwidth, computing resources, etc.

Because a large amount of data is stored on each node (typically many terabytes), and all data comprising fragments stored on a node typically needs to be replaced when the node permanently fails, the repair process reads and writes a large amount of data after a node permanently fails.

This reading and writing of large amounts of data generally results in the consumption of large amounts of bandwidth in the storage system by the repair policy and, when coupled with a reactive repair policy, can result in disruptive spikes in storage system bandwidth utilization (e.g., delaying or preventing primary data access services by the storage system).

That is, the repair policy implemented with respect to a small erasure code configuration may act somewhat erratically whereby it utilizes a large amount of bandwidth for periods of time when attempting to repair data immediately upon determining fragments are unavailable due to a permanently failed node, whereas at other periods of time the repair policy may utilize no bandwidth.

Furthermore, the repair process may not be able to repair fragments fast enough before further node failures lead to permanent loss of object data if not enough bandwidth is allocated to the repair process.

Images