Wan-optimized local and cloud spanning deduplicated storage system

Abstract

A spanning storage interface facilitates the use of cloud storage services by storage clients. The spanning storage interface presents one or more data interfaces to storage clients at a network location, such as file, object, data backup, archival, and storage block based interfaces. The data interfaces allows storage clients to store and retrieve data using non-cloud based protocols. The spanning storage interface may perform data deduplication on data received from storage clients. The spanning storage interface may transfer the deduplicated version of the data to the cloud storage service. The spanning storage interface may include local storage for storing a copy or all or a portion of the data from storage clients. The local storage may be used as a local cache of frequently accessed data, which may be stored data in its deduplicated form.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims priority to U.S. Provisional Patent Application No. 61 / 315,392, filed Mar. 18, 2010 and entitled “WAN-OPTIMIZED LOCAL AND CLOUD SPANNING DEDUPLICATED STORAGE SYSTEM” and to U.S. Provisional Patent Application No. 61 / 290,334, filed Dec. 28, 2009 and entitled “DEDUPLICATED OBJECT STORAGE SYSTEM AND APPLICATIONS,” which are incorporated by reference herein for all purposes. This application is related to U.S. patent application Ser. No. ______ [Docket Number R001520US], filed ______, and entitled “DISASTER RECOVERY USING LOCAL AND CLOUD SPANNING DEDUPLICATED STORAGE SYSTEM,” which is incorporated by reference herein for all purposes.BACKGROUND OF THE INVENTION[0002]The present invention relates generally to data storage systems, and systems and methods to improve storage efficiency, compactness, performance, reliability, and compatibility. In general, data storage systems receive and store all or portions of arbitrary...

AI Technical Summary

Benefits of technology

[0016]Embodiments of the invention include a spanning storage interface adapted to facilitate the use of cloud storage services by storage clients. A spanning storage interface presents one or more data interfaces to storage clients at a network location. These data interfaces may include file, object, data backup, archival, and storage block based interfaces. Each of these data interfaces allows storage clients to store and retrieve data using non-cloud based protocols. This allows storage clients to store and retrieve data in the cloud storage service using their native or built-in functions, rather than having to be rewritten and / or reconfigured to operate with a cloud storage service.

[0017]To improve performance of the spanning storage interface, an embodiment of the invention performs data deduplication on data received from storage clients. Once the received data has been deduplicated, the spanning storage interface may transfer the deduplicated version of the data to the cloud storage service. By transferring data in deduplicated form to and from the cloud storage service, these embodiments of the invention improve storage performance by reducing the time and network bandwidth required to access data, as well as reducing total amount of storage required. If a storage client wishes to access data previously stored in the cloud storage service, the spanning storage interface retrieves the corresponding deduplicated data and reconstructs the original data.

[0020]In an embodiment, two or more spanning storage interfaces may be used in a disaster recovery application. Disaster recovery application may be used to provide redundant data access to storage clients in the event that the storage clients and / or cloud spanning storage interface at a first network location are disabled, destroyed, or otherwise inaccessible or inoperable. A disaster recovery application includes at least first and second spanning storage interfaces at first and second network locations. The second spanning storage interface is provided for at least disaster recovery operations. The second spanning storage interface includes second local storage for improving data access performance. A copy of the local cache of the first spanning storage interface is transferred to the second local storage while the first network location is operating. In the event of a disaster affecting the first network location, the second spanning storage interface can provide data access to the first network location's data with the improved performance benefit using the copy of local cache in the second local storage.

[0021]Embodiments of the disaster recovery application may use the second network location as a dedicated disaster recovery network location. Alternatively, the second network location may also optionally be used with one or more of its own local storage clients. In this further example, the second spanning storage interface performs data deduplication and facilitates cloud storage for data from storage clients at the second network location in addition to acting as a disaster recovery system for the first network location. In yet a further embodiment, the first spanning storage interface may act as a disaster recovery system for the second spanning storage interface, just as the second spanning storage interface may act as a disaster recovery system for the first spanning storage interface. This pairing of spanning storage interfaces for disaster recovery may be extended to three or more network locations.

Problems solved by technology

The CPU and I / O requirements for supporting an extremely large data deduplicating storage are significant, and are difficult to satisfy through vertical scaling of a single device.

As a result, prior spanning storage interface may impose severe throughput, latency, and other performance penalties on storage clients.

Additionally, performance considerations limit the amount and types of optimizations and compression applied by prior spanning storage interfaces.

Additionally, prior spanning storage interfaces have difficulty operating with cloud storage systems.

However, cloud data storage is accessible only via a wide-area network, such as the Internet, with significant latency and bandwidth limitations as compared with local-area and storage-area networks.

Therefore, prior spanning storage interfaces have poor performance when used with cloud storage systems.

Images