Device driver deployment of similarity-based delta compression for use in a data storage system

Abstract

A data storage caching architecture supports using native local memory such as host-based RAM, and if available, Solid State Disk (SSD) memory for storing pre-cache delta-compression based delta, reference, and independent data by exploiting content locality, temporal locality, and spatial locality of data accesses to primary (e.g. disk-based) storage. The architecture makes excellent use of the physical properties of the different types of memory available (fast r / w RAM, low cost fast read SSD, etc) by applying algorithms to determine what types of data to store in each type of memory. Algorithms include similarity detection, delta compression, least popularly used cache management, conservative insertion and promotion cache replacement, and the like.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application is a continuation of U.S. patent application U.S. Ser. No. 13 / 366,846 entitled Pre-Cache Similarity-Based Delta Compression for use in a Data Storage System, filed Feb. 6, 2012, which is hereby incorporated herein by reference in its entirety.[0002]U.S. patent application U.S. Ser. No. 13 / 366,846 claims the benefit of the following provisional applications, each of which is hereby incorporated herein by reference in its entirety: U.S. Ser. No. 61 / 441,976 entitled Intelligently Coupled Array of SSD and HDD, filed Feb. 11, 2011; U.S. Ser. No. 61 / 447,208 entitled Effective Page Classification and Delta Encoding AND Buffer Cache—Temporal and Content Localities, filed Feb. 28, 2011; and U.S. Ser. No. 61 / 497,549 entitled Conservative Insertion and Promotion Cache Replacement Algorithm, filed Jun. 16, 2011.[0003]U.S. patent application U.S. Ser. No. 13 / 366,846 is a continuation-in-part of U.S. patent application U.S. Ser. No. 12...

AI Technical Summary

Benefits of technology

This approach enhances I / O performance, extends SSD life, reduces power consumption, and improves reliability by minimizing random writes to SSDs and leveraging high-speed computations, while also reducing storage costs through efficient data redundancy elimination and compression.

Problems solved by technology

While data storage capacity and central processing unit (CPU) processing power have experienced rapid growth in the past, improvement in data bandwidth and access times of disk input / output (I / O) systems have not kept pace.

As a result, there is an ever-widening speed gap between CPU and disk I / O systems.

Disk arrays can improve overall I / O throughput, but random access latency is still very high because of mechanical operations involved.

Large buffers and deep cache hierarchy can improve latency but the resulting access time reduction is very limited.

However, limitations that result from the physical properties of SSDs include write and erase times that are very slow compared to read times, and a significantly limited number of write / erase cycles before failure of a block.

The above physical properties and operational characteristics of SSDs present unique challenges in designing a SSD-based mass storage hierarchy.

Also, while the capacity of disk drives grows rapidly, their electromechanical parts have held down the improvement of their performance.

This may result in much more data redundancy than would be present when running a single OS on a data center server.

Images