Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Migration management based on massive data classified memory system

A technology for hierarchical storage and massive data, applied in memory systems, electrical digital data processing, special data processing applications, etc. It can solve problems such as large burst traffic, no consideration of the impact of migration on front-end applications, and lack of adaptive mechanism for degradation. , to achieve the effect of increasing the ratio

Inactive Publication Date: 2008-01-09
TSINGHUA UNIV
View PDF0 Cites 83 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Traditional tiered storage systems seldom include migration management and rate control mechanisms. If a file is accessed on a slow data server, it will be migrated to a fast data server without considering the impact of migration on front-end applications.
At the same time, the traditional tiered storage system lacks an adaptive mechanism for downgrading. If the disk space of the fast data server is full or its remaining space is lower than a certain threshold, the files on the fast data server will be downgraded, resulting in a relatively large burst of system traffic. Large, when the remaining disk space is lower than the allowable value, if you want to upgrade a file, you must first downgrade a file from the fast data server to free up free storage space, which increases the upgrade cost

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Migration management based on massive data classified memory system
  • Migration management based on massive data classified memory system
  • Migration management based on massive data classified memory system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] The mass data hierarchical storage system is mainly composed of metadata server, data server and parallel file system client agent software on the front-end host. The migration management method runs in the massive hierarchical storage system.

[0054] In the massive hierarchical storage system, the front-end host, the metadata server and the data server are connected together through an Ethernet switch. According to the performance of the devices mounted on the data server, it is divided into fast and slow data servers. The migration start end of the migrated files is called the source data server, and the migration destination end is called the target data server.

[0055] The metadata service module on the metadata server completes the metadata operation, and the incremental scanning module periodically sends scanning instructions to all data servers to obtain the access status of the data servers in this period; the migration management module maintains and updates ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Characters of the method are that client proxy software of parallelized file system on each front-end host implements supporting VFS access; server of metadata is in charge of executing increment scaling, transporting management and rate control i.e. operations so as to realize performance guarantee of front-end application in hierarchical storage system for mass data (HSSMD). According to performance, data servers are divided into fast data servers and slow data servers. Data server processes incremental scan command sent from metadata server, returns scanned information, and meanwhile executes file transfer command sent from metadata server. Based on condition of load, the method dispatches procedure of data transfer. Based on condition of remnant space (RS) of fast device, the method carries out degradation initiatively. The invention reduces influence on application in front end from transporting procedure, raises rate of RS of fast device, and enhances self management for HSSMD.

Description

technical field [0001] A migration management method based on a massive data hierarchical storage system belongs to the field of data migration, and particularly relates to the fields of incremental scanning and rate control therein. Background technique [0002] The migration management method based on the massive data hierarchical storage system refers to: in the massive data hierarchical storage system, the data is classified according to the access rules of the massive data, and the storage subsystems of different levels are divided according to the change of the data level and the usage of the equipment. Migrate data between and manage migration tasks to reduce their impact on front-end applications. Traditional tiered storage systems seldom include migration management and rate control mechanisms. If files are accessed on slow data servers, they will be migrated to fast data servers without considering the impact of migration on front-end applications. At the same tim...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50G06F17/30G06F12/08G06F12/123
Inventor 舒继武陈康于得水张广艳
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com