Client origin information associative perception based metadata pre-acquisition method and system

A technology of origin information and metadata, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of not applying metadata semantic information, not considering historical process behavior information, etc., to reduce request response time , reduce error association calculations, and avoid the effect of disk I/O

Active Publication Date: 2016-01-27
JINAN UNIVERSITY
View PDF2 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The literature [ANovelWeighted-Graph-BasedGroupingAlgorithmforMetadataPrefetching] uses a movable history window to conduct correlation statistics on historical access sequences, store them in the graph data structure, and perform metadata prefetching, which improves the I / O performance of metadata services; however, This method simply analyzes the historical access patterns of file I / O, and does not apply metadata-rich semantic information
In addition, the literature [FARMER: novel approach to file access correlation mining and evaluation reference model for optimizing peta-scale file system performance] calculates the file semantic distance, combined with the historical access sequence, effectively improves the accuracy of prefetching, but this method only calculates the similarity of file attributes, and does not take into account the file being manipulated Historical process behavior information, that is, the origin information of the client

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Client origin information associative perception based metadata pre-acquisition method and system
  • Client origin information associative perception based metadata pre-acquisition method and system
  • Client origin information associative perception based metadata pre-acquisition method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0049] Such as figure 1 As shown, the metadata prefetching system of client origin information association awareness in this embodiment includes an origin information collection module and an association score calculation module, wherein:

[0050] The source information collection module is designed according to the service architecture of the client and is installed on the client for real-time collection of source information log records in the client's kernel space, using the Netlink protocol to transmit from the kernel space to the user space and store the source information In the database; wherein, the origin information includes the process origin information of the process start and end time and the I / O (Input / Output, input / output) request origin information of the process operation file;

[0051] The correlation score calculation module is used to select a part of the collected origin information log records (2-day origin information log records) on the client side, as...

Embodiment 2

[0054] Such as figure 2 As shown, the metadata prefetching method of client origin information association perception in this embodiment is implemented based on the system in Embodiment 1, and includes the following steps:

[0055] S1. The origin information collection module collects origin information log records in real time in the kernel space of the client, transmits them from the kernel space to the user space using the Netlink protocol, and stores them in the origin information database; wherein, the origin information includes process start and end times Process origin information and I / O request origin information of process operation files;

[0056] The source information log records are collected in real time in the kernel space of the client, specifically:

[0057] In the kernel space of the client, intercept the exit and exit_group system calls, collect the process origin information log records of the process start and end time; intercept the open, read, write,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a client origin information associative perception based metadata pre-acquisition method and system. The method comprises: collecting origin information log records in a kernel space of a client, and transmitting the records to a user space from the kernel space; selecting part of the collected origin information log records at the client as associative training data of metadata, and computing an associative score between every two metadata to obtain an original strong associative list; when the client initiates a file access request and metadata of a file are not in a local metadata cache of the client, pre-acquiring multiple strongly associative metadata from the strong associative list of the client, downloading corresponding metadata from a metadata server, and updating the local metadata cache of the client; and according to added origin information log records, regularly updating the strong associative list of the metadata. According to the method and the system, the metadata cache hit rate of the client is increased, and the frequency of access to the metadata server is reduced, so that the performance of the metadata server is improved.

Description

technical field [0001] The present invention relates to a metadata prefetching method and system, in particular to a metadata prefetching method and system based on client origin information association perception, which belongs to the mining technology of origin information collection and metadata correlation and metadata prefetching technology field. Background technique [0002] With the continuous growth of data, the data volume of the storage system in the high-performance computing environment becomes larger and larger, and the data storage volume reaches the TB level or even the PB level. For example, Facebook already has 200M data objects, occupying 21PB of storage space. In order to improve the I / O performance of the storage system, most distributed file systems today usually separate file data and metadata, that is, data flow and control flow, so as to obtain higher system scalability and I / O concurrency. Metadata is stored separately in one or more metadata serv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/182
Inventor 邓玉辉吴国锦
Owner JINAN UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap