Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method, device, system and equipment for metadata migration

A metadata and technology to be migrated, applied in the computer field, can solve the problems of low efficiency, time-consuming, and uneven data volume of migration tasks, etc., and achieve the effect of improving overall work efficiency and shortening the time spent

Active Publication Date: 2022-05-03
ALIBABA GRP HLDG LTD
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] After researching the existing technology, the inventor found that in the process of metadata migration between metadata server groups, the migration task is generated based on the number of files contained in the entry to be migrated. Due to the uneven granularity of the files, the generated migration The amount of data contained in the task is uneven, which may cause some execution servers to be assigned migration tasks with a large amount of data, making it take a lot of time for the execution server to complete the migration task, resulting in low overall work efficiency of metadata migration

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device, system and equipment for metadata migration
  • Method, device, system and equipment for metadata migration
  • Method, device, system and equipment for metadata migration

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0037] Such as figure 1 As shown, the embodiment of the present application provides a method for metadata migration. The execution subject of the method may be a master server (also called a Master server) that controls metadata migration. The master server controls multiple execution servers (also It is called the Work server), and performs metadata migration between multiple metadata server groups to balance the load of each metadata server group. The method specifically may include the following steps:

[0038] Step S101: Determine the entries to be migrated in the directory to be migrated.

[0039] Wherein, the directory to be migrated may be a directory where metadata migration needs to be performed, that is, a directory where metadata to be migrated is located. Metadata can be information used to describe data attributes, and is often used to support functions such as indicating the storage location of data, searching for historical data or resources, and document rec...

Embodiment 2

[0053] The embodiment of the present application provides a metadata migration method. The execution body of the method can be a master server. The master server performs metadata migration among multiple metadata server groups by controlling multiple execution servers, so as to balance the The load on the metadata server group. The method specifically may include the following steps:

[0054] The processing manner of S101 in the first embodiment above may be varied, and a specific processing manner is provided below, which may include the following steps S201 to S203.

[0055] In step S201: traverse from the root directory of the directory tree where the directory to be migrated is located, and record the currently traversed entry identifier during the traverse process.

[0056] Among them, the entry can include directories and files, where the directories can include root directories and subdirectories, and the root directory can be the top-level directory where files are s...

Embodiment 3

[0104] The above is the metadata migration method provided by the embodiment of the present application. Based on the same idea, the embodiment of the present application also provides a metadata migration device, such as Figure 4 shown.

[0105] The metadata migration device includes: an item to be migrated determination module 401, a migration task generation module 402, and a migration task assignment module 403, wherein:

[0106] The item to be migrated determination module 401 is used to determine the item to be migrated in the directory to be migrated;

[0107] A migration task generating module 402, configured to generate a migration task according to the number of data blocks corresponding to the entry to be migrated, and divide the entry to be migrated for the migration task;

[0108] The migration task allocation module 403 is configured to allocate the migration task to an execution server to perform metadata migration on the item to be migrated.

[0109] In the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the present application discloses a metadata migration method, device, system, and equipment. The method includes: determining the entry to be migrated in the directory to be migrated; generating a migration task according to the number of data blocks corresponding to the entry to be migrated, dividing the entry to be migrated for the migration task; assigning the migration task to an execution server to perform metadata migration on the entry to be migrated. Utilizing the embodiment of the present application, it is possible to achieve a balanced distribution of items to be migrated in each migration task, thereby avoiding the execution server that may be assigned a migration task with a large amount of data, and shortening the time it takes for the execution server to complete the migration task, thereby Can improve the overall productivity of metadata migration.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular to a metadata migration method, device, system and equipment. Background technique [0002] At present, large-scale distributed storage systems are widely used. In distributed storage systems, the metadata server manages the file system namespace and various attributes of files, and provides information such as file access rights and file storage locations. At the same time, the data storage server stores files and processes them. Various read and write requests for file data by the client. Metadata servers in most distributed storage systems are usually composed of multiple groups to provide services, and each group of servers constitutes a metadata server group (that is, volume). Distributed storage systems are usually configured with multiple metadata server groups. The data pressure is distributed among metadata server groups. [0003] During the operation of t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/11G06F16/182G06F9/50
CPCG06F16/1824G06F16/119G06F9/505G06F2209/5015G06F9/48G06F9/50G06F16/00
Inventor 吕鹏程姚文辉刘俊峰黄硕朱家稷
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com