Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data multiplexing method and device, electronic equipment and storage medium

A data multiplexing and data writing technology, applied in the computer field, can solve problems such as large memory overhead

Pending Publication Date: 2019-11-05
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

By caching the written data in memory, it can solve the problem of disk IO twice for reading and writing request data, but it also brings greater memory overhead.
At the same time, if there is a high probability of a master-slave replica network partition, slave replica machine downtime and other abnormal conditions, a full synchronization needs to be triggered to ensure data consistency between the master and slave, which brings additional network bandwidth overhead

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data multiplexing method and device, electronic equipment and storage medium
  • Data multiplexing method and device, electronic equipment and storage medium
  • Data multiplexing method and device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0053] figure 1 It is a schematic flow chart of the data multiplexing method provided in Embodiment 1 of the present invention. The method can be executed by a data multiplexing device or electronic equipment. The device or electronic device can be implemented by software and / or hardware. The device or electronic The device can be integrated in any smart device with network communication function. Such as figure 1 As shown, the data multiplexing method may include the following steps:

[0054] S101. Using the synchronous log module to store the write request sent by the user; wherein, the write request includes: currently written data and a serial number of the currently written data.

[0055] In a specific embodiment of the present invention, the electronic device can store the write request sent by the user through the synchronous log module; wherein, the write request includes: the current write data and the serial number of the current write data. Specifically, the dist...

Embodiment 2

[0067] Figure 5 It is a schematic flowchart of the data multiplexing method provided in Embodiment 2 of the present invention. Such as Figure 5 As shown, the data multiplexing method may include the following steps:

[0068] S501. Using the synchronous log module to store the write request sent by the user; wherein, the write request includes: currently written data and a serial number of the currently written data.

[0069] In a specific embodiment of the present invention, the electronic device can store the write request sent by the user through the synchronous log module; wherein, the write request includes: the current write data and the serial number of the current write data. Specifically, the distributed storage system may include a plurality of electronic devices, which are respectively: electronic device 1, electronic device 2, ... electronic device N; wherein, N is a natural number greater than 1. Each electronic device may include: a synchronous log module and...

Embodiment 3

[0089] Figure 7 It is a schematic diagram of the first structure of the data multiplexing device provided by Embodiment 3 of the present invention. Such as Figure 7 As shown, the data multiplexing device described in the embodiment of the present invention may include: a first storage module 701, a second storage module 702, a first establishment module 703, and a second establishment module 704; wherein,

[0090] The first storage module 701 is configured to store the write request sent by the user through the synchronous log module; wherein, the write request includes: the currently written data and the serial number of the currently written data;

[0091] The second storage module 702 is configured to store the currently written data into the current data file of the storage engine module, and hard-link the current shadow file corresponding to the current data file;

[0092] The first establishing module 703 is configured to establish the first index information of the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a data multiplexing method and device, electronic equipment and a storage medium. The method comprises the following steps: storing a write-in request sent bya user through a synchronous log module; wherein the write-in request comprises current write-in data and a serial number of the current write-in data; storing the current write-in data into a current data file of a storage engine module, and performing hard link on the current data file to obtain a current shadow file corresponding to the current data file; establishing first index information of the currently written data in a synchronous log module according to the number of the currently written data and the current shadow file; and establishing second index information for the current write-in data in the storage engine module after the current write-in data is submitted. According to the embodiment of the invention, the synchronous log module and the storage engine module can reuseone piece of data, the writing pressure of a disk is reduced under the same writing QPS per second, the writing performance of a distributed storage system is improved, and the service life of storageequipment is prolonged.

Description

technical field [0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a data multiplexing method, device, electronic equipment, and storage medium. Background technique [0002] A distributed storage system stores data dispersedly on multiple independent devices. Traditional network storage systems use centralized storage servers to store all data. Storage servers become the bottleneck of system performance and the focus of reliability and security, which cannot meet the needs of large-scale storage applications. The distributed network storage system adopts a scalable system structure, uses multiple storage servers to share the storage load, and uses the location server to locate and store information. It not only improves the reliability, availability and access efficiency of the system, but also is easy to expand. [0003] A distributed storage system usually involves a synchronous log module and a storage engine mo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/11G06F16/13G06F16/172G06F16/18G06F16/182
CPCG06F16/122G06F16/134G06F16/172G06F16/1815G06F16/1824G06F16/1847
Inventor 赖梓昌田勇李理
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com