Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Distributed file system metadata management method facing to high-performance calculation

A technology for distributed files and file metadata, which is used in electrical digital data processing, special data processing applications, computing, etc.

Active Publication Date: 2013-06-12
NAT UNIV OF DEFENSE TECH
View PDF3 Cites 107 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The technical problem to be solved by the present invention is to provide a high-performance computing-oriented system with high storage performance, low maintenance overhead, no bottleneck for high load, good scalability, load balance, and effective solution to the metadata migration problem caused by directory renaming. Distributed File System Metadata Management Method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system metadata management method facing to high-performance calculation
  • Distributed file system metadata management method facing to high-performance calculation
  • Distributed file system metadata management method facing to high-performance calculation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0103] like figure 1 As shown, the implementation steps of the high-performance computing-oriented distributed file system metadata management method in this embodiment are as follows:

[0104] 1) Establish directory metadata server clusters for managing directory metadata, file metadata server clusters for managing file metadata, and establish object interface data storage services for directory metadata server clusters and file metadata server clusters object storage server cluster;

[0105] 2) Establish Bloom filters in all directory metadata server clusters, and share Bloom filters among directory metadata servers in the directory metadata server cluster;

[0106] 3) When the operation request from the client arrives, if the operation request is a directory operation request, skip to step 4); if the operation request is a file operation request, skip to step 5);

[0107] 4) The directory metadata server cluster responds to the client's directory operation request to mana...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed file system metadata management method facing to high-performance calculation. The method comprises the following steps of: 1) establishing a catalogue metadata server cluster, a file metadata server cluster and an object storage server cluster; 2) establishing a global counting-based bloom filter in the catalogue metadata server cluster; 3) when the operation request of a client side arrives, skipping to execute step 4) or 5); 4) enabling the catalogue metadata server cluster to respond to the catalogue operation request of the client side to manage the catalogue metadata; and 5) enabling the file metadata server cluster to respond to the file operation request of the client side to manage the file metadata data. According to the distributed file system metadata management method disclosed by the invention, the metadata transferring problem brought by catalogue renaming can be effectively solved, and the distributed file system metadata management method has the advantages of high storage performance, small maintenance expenditure, high load, no bottleneck, good expansibility and balanced load.

Description

technical field [0001] The invention relates to the field of distributed file system architecture, in particular to a high-performance computing-oriented distributed file system metadata management method. Background technique [0002] With the demands of various I / O-intensive applications and the development of high-speed networks, the scale of today's high-performance computer cluster computing nodes has reached tens of thousands, and the performance has reached PFlops. The scale of the underlying storage system continues to increase, and PB-level or even larger systems continue to appear. The number of files in a distributed file system can reach hundreds of millions, and it is an urgent problem to quickly and efficiently handle the concurrent access to file metadata by thousands of clients. The metadata server (Metadata Server, MDS) is used to manage the metadata of the file system. When the scale of the storage system is very large, one MDS will become the performance ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30H04L29/08
Inventor 卢宇彤王立新肖立权周恩强肖侬所光张伟董勇陈海涛
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com