Flash memory cache including for use with persistent key-value store

Abstract

Described is using flash memory, RAM-based data structures and mechanisms to provide a flash store for caching data items (e.g., key-value pairs) in flash pages. A RAM-based index maps data items to flash pages, and a RAM-based write buffer maintains data items to be written to the flash store, e.g., when a full page can be written. A recycle mechanism makes used pages in the flash store available by destaging a data item to a hard disk or reinserting it into the write buffer, based on its access pattern. The flash store may be used in a data deduplication system, in which the data items comprise chunk-identifier, metadata pairs, in which each chunk-identifier corresponds to a hash of a chunk of data that indicates. The RAM and flash are accessed with the chunk-identifier (e.g., as a key) to determine whether a chunk is a new chunk or a duplicate.

Description

BACKGROUND[0001]Flash media has advantages over RAM and hard disk storage, namely that unlike RAM, flash media is persistent, and unlike hard disk, flash media provides much faster data access times, e.g., on the order of hundreds or thousands of times faster than hard disk access. Many applications thus may benefit from the use of flash media.[0002]However, flash media is expensive, at present costing ten to twenty times more per gigabyte than hard disk storage. Further, flash devices are subject to reduced lifetimes due to page wearing, whereby small random writes (that also have relatively high latency) are not desirable. What is needed is a technology for using flash media that provides high performance, while factoring in cost considerations, efficiency and flash media lifetimes.SUMMARY[0003]This Summary is provided to introduce a selection of representative concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended ...

AI Technical Summary

Benefits of technology

This patent describes a technology that allows a flash memory to be used as a cache between RAM and a hard disk store. The technology uses RAM-based data structures and mechanisms to maintain a cache of data items in flash pages. The flash store can be used in conjunction with RAM in a data deduplication system, where data items are chunked and prefetched into the RAM cache or added to a container in the flash store. The technology allows for efficient data storage and retrieval, improving the performance and speed of the system.

Problems solved by technology

However, flash media is expensive, at present costing ten to twenty times more per gigabyte than hard disk storage.

Further, flash devices are subject to reduced lifetimes due to page wearing, whereby small random writes (that also have relatively high latency) are not desirable.

Images