Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Small file access method accelerated based on solid state disk for distributed file system

A distributed file and access method technology, applied in the field of distributed file system small file access, can solve problems affecting small file access performance and efficiency, occupying data server IO resources, occupying small file reading time, etc., to save communication Effects of delay, high IO performance, and calculation time saving

Active Publication Date: 2017-05-31
NAT UNIV OF DEFENSE TECH
View PDF4 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this way, even when users access small files, they need to perform multiple IO requests, and the high network delay and multiple disk IO delays take up a lot of time for reading small files, which affects the access performance and efficiency of small files
There are two problems in the access of small files in the current distributed file system: 1) The data volume of small files is very small, and it needs to communicate with the data server network after obtaining metadata, which takes up network delay; 2) In order to obtain the actual data of small files File data requires one or more disk IOs with the object server, which not only increases disk IO overhead, but also occupies data server IO resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Small file access method accelerated based on solid state disk for distributed file system
  • Small file access method accelerated based on solid state disk for distributed file system
  • Small file access method accelerated based on solid state disk for distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] The following will take the SSD-based DFS as an example to describe the small file access method of the SSD-based distributed file system in detail. The difference between SSD-based DFS and general distributed file systems is that solid-state disks are used as storage media. For example, in this embodiment, the file metadata area of ​​the metadata server of SSD-based DFS uses solid-state disks as storage media.

[0044] Such as image 3 As shown, the steps of the distributed file system small file access method based on solid state disk acceleration in this embodiment include:

[0045] 1) Set the embedded file size threshold in the distributed file system;

[0046] 2) Receive the type of user request through the client (Client) of the distributed file system, if the type is to create a file, then jump to step 3); if the type is to access a file, then jump to step 4);

[0047] 3) The client compares the size of the created file with the embedded file size threshold, an...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a small file access method accelerated based on a solid state disk for a distributed file system. The small file access method comprises the following steps of setting the threshold value of the size of an embedded file; when a client establishes a file, if the size of the established file is less than the threshold value of the size of the embedded file, embedding and storing the established file in a file metadata area, which is located in the solid state disk, of a metadata server as the embedded file, otherwise, storing the established file in an object storage server; when a process of the client opens the file, according to a storage position in file metadata information, responding to the access request of a user and returning an execution result by the metadata server or the object storage server. According to the small file access method, the data of a small file are directly stored in a metadata area of the solid state disk; the computing time of the storage position of the data of the small file is saved; the access communication time delay of the small file is saved; the access performance of the small file is promoted; the small file is conveniently optimized; the load of the metadata server cannot be aggravated; the overall performance of the distributed file system cannot be decreased.

Description

technical field [0001] The invention relates to the technical field of distributed file system data processing, in particular to a method for accessing small files in a distributed file system based on solid-state disk acceleration. Background technique [0002] With the rapid development of cloud computing and big data technology, the amount of global data is growing explosively, which brings new development opportunities and challenges for distributed file systems, from high-performance computing to data centers, from data sharing to Internet applications , the distributed file system has penetrated into all aspects of data applications. Most of the current distributed file systems are for the processing of large-scale big data files, and the storage optimization of small files is relatively lacking. With the development of social networks, mobile communications and personal terminals, the proportion of small files in the storage system is getting higher and higher. For ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0611G06F3/0629G06F3/0643G06F3/0655G06F3/067
Inventor 汪黎文云川肖先霞吴庆波戴华东谭郁松杨沙洲
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products