Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

A Big Data Backup Method Based on Virtual Shared Directory

A virtual sharing, big data technology, applied in the redundancy of computing for data error detection, electrical digital data processing, response error generation and other directions, can solve the problem of different internal storage protocols, increase the risk of system operation, data backup Efficiency is difficult to guarantee and other problems, to achieve the effect of improving compatibility, simplifying complexity, and improving compatibility

Active Publication Date: 2020-06-23
吉林吉大通信设计院股份有限公司
View PDF13 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] In the prior art (1), a corresponding collection client is required for different backup objects, and an agent is required to transfer data from a real data source (such as hadoop namenode) to a local temporary storage directory (on the host), Then the data in this directory is processed by cutting into blocks (for example, one 64K data block each time), and then each data block is transmitted to the media server through the HTTP protocol. After the media server receives it, it undergoes a series of deduplication and compression processing Finally, the data will be transmitted to a special storage medium (such as disk) through the FC network through the ISCSI protocol. The entire process data will go through four key time-consuming steps (namely, agent local temporary storage, local switching, network transmission to the media server, Media server network transmission to storage media), the efficiency of data backup is difficult to be guaranteed, and too many links also increase the operating risk of the system
[0010] Compared with technology (1), the difference is that after the data is transmitted to the media server, the data is not directly transmitted to the storage medium through the ISCSI protocol, but is cut into blocks again through the HTTP protocol, and the data is transmitted to the object storage through the HTTP protocol Medium (in object storage), technology (2) is different from technology (1) only in the back-end internal storage protocol, and the overall storage efficiency and risks are not effectively avoided. At the same time, it is aimed at the collection of multiple types of big data platforms It is also necessary to develop the corresponding client agent agent, and the complexity and compatibility of the backup system have not yet been improved.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Big Data Backup Method Based on Virtual Shared Directory

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] A kind of big data backup method based on virtual shared directory is characterized in that: comprise the following steps:

[0028] Step 1. Establish a virtual shared data storage backup system including big data platform, backup media layer, media service layer and storage media;

[0029] Step 2: The big data platform initiates a backup request to the system, and the backup medium layer remotely mounts the network file medium NFS agent on the big data platform, provides a virtual shared directory based on the network file NFS protocol for the big data platform, and temporarily stores the data to the internal directory of the NFS agent;

[0030] Step 3: After the temporary storage of the NFS agent provided by the backup medium layer is completed, the virtual sharing link is disconnected, and the data of the big data platform belongs to the backup medium layer;

[0031] Step 4: After the backup medium layer performs data processing, the NFS agent is sent to the storage ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A big data backup method based on a virtual shared directory relates to the technical field of data backup. Through local storage on a media server, a file sharing protocol interface is provided, a virtual shared directory will be created, if the interface is provided to a big data platform A that needs to be backed up, when the big data platform A needs to be backed up, then the sharing right ofthe virtual directory can be obtained by mounting the partition locally. After the backup is finished, the partition is disconnected, and the partition can fall back to the media server. At the same time, the partition can provide the shared directory service to another storage server. Through file replication, the backup of the big data file is realized very concisely.

Description

technical field [0001] The invention belongs to the technical field of data backup, and in particular relates to a big data backup method for improving the efficiency of big data backup. Background technique [0002] The value of data in the era of big data is more critical, and the security of data running on big data needs to be guaranteed, so a faster and more general backup technology is needed to realize data backup of various big data platforms and ensure backup efficiency and compatibility. [0003] At present, data backup methods for big data platforms generally follow the following architecture, which includes the following parts: backup agent (ie agent), media server, and storage media. [0004] The specific implementation details can be roughly divided into the following two types: [0005] (1) client agent HTTP media server iSCSI storage medium [0006] The backup agent is installed on the big data host to be backed up, collects the backup data, and tr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/14
CPCG06F11/1461G06F11/1464G06F11/1469
Inventor 匙凯于富东胡建华杨林崔明阳
Owner 吉林吉大通信设计院股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products