Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data processing method and data processing device

A data processing device and data processing technology, applied in the storage field, can solve the problems of poor performance and the inability to take into account the speed of writing data at the same time, and achieve the effects of pure continuous writing, fast persistence, and improved IOPS performance

Active Publication Date: 2014-06-04
HUAWEI CLOUD COMPUTING TECH CO LTD
View PDF4 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But adopting the above-mentioned method of writing data cannot simultaneously take into account the speed of writing data from NVM to the log area of ​​the hard disk and the speed of sorting out the written data in the log area of ​​the hard disk
[0005] Specifically, if you want to ensure the speed of writing data to the log area of ​​the hard disk, you may sacrifice the speed of data sorting; if you want to ensure the speed of data sorting, you may sacrifice the speed of writing data to the log area of ​​the hard disk. The number of read and write operations per second (Input / Output Operations Per Second, IOPS) during processing is poor

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and data processing device
  • Data processing method and data processing device
  • Data processing method and data processing device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] An embodiment of the present invention provides a data processing method, such as figure 1 shown, including:

[0056] S101. If the used log space of the log area of ​​the first hard disk group is higher than the preset threshold, the data processing device writes the first data read from the internal memory and the characteristic value of the first data into the log of the second hard disk group area, and suspend the collation of the data written in the log area of ​​the second hard disk group to the first data area.

[0057] Wherein, the data written in the log area of ​​the second hard disk group includes the first data.

[0058] It should be noted that the data processing device in the embodiment of the present invention may be a data control module or a data processing module in the database, which is used to control writing and reading of data in the database. For example, the data control module may be a central processing unit (Central Processing Unit, CPU) of ...

Embodiment 2

[0084] An embodiment of the present invention provides a data processing method, taking two hard disk groups (the data areas of the first hard disk group and the second hard disk group) respectively performing log writing and data sorting as an example, in this embodiment, as Image 6 As shown, the first hard disk group includes the log area and the data area, and the second hard disk group includes the log area and the data area; the first data area is the data area of ​​the second hard disk group, and the second data area is the data area of ​​the first hard disk group ;like figure 2 As shown, the method of this embodiment may include:

[0085] S201. A process in which the data processing device writes data to be written into a memory.

[0086] It should be noted that the memory in this embodiment of the present invention may include: a first memory unit and a second memory unit. Wherein, for the specific description of the first memory unit and the second memory unit, re...

Embodiment 3

[0167] The embodiment of the present invention provides a data processing method. Two hard disk groups (the first hard disk group and the second hard disk group) are used for log writing and data arrangement respectively, and the other hard disk group (the third The data obtained by hard disk group arrangement is taken as an example. In the embodiment of the present invention, such as Figure 7 As shown, the first hard disk group only includes the log area, and the second hard disk group only includes the log area; the first data area and the second data area are both data areas of the third hard disk group; image 3 As shown, the method of this embodiment may include:

[0168] S301. A process in which the data processing device writes data to be written into a memory.

[0169] It should be noted that the memory in this embodiment of the present invention may include: a first memory unit (such as NVM) and a second memory unit (such as DRAM). Wherein, for the specific descrip...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

An embodiment of the invention discloses a data processing method and a data processing device, and relates to the technical field of storage. By the data processing method and the data processing device, data writing speed and data arrangement speed can be increased simultaneously, IOPS (input / output operation per second) performance of the data processing process is improved, and further, rapidness and persistence of data can be realized. The data processing method specifically includes if the occupied quantity of a log space of a log area of a first hard-disk pack is higher than a preset threshold value, writing first data and feature values thereof both read from an internal storage into a log area of a second hard-disk pack, and pausing arranging data written in the log area of the second hard-disk pack into a first data area; according to the data feature values written in the log area of the first hard-disk pack, arranging the data written in the log area of the first hard-disk pack to a second data area. The data processing method and the data processing device are applied to data read-write processes.

Description

technical field [0001] The invention relates to the field of storage technology, in particular to a data processing method and device. Background technique [0002] With the rapid development of processor technology, existing storage systems are not enough to match the rapidly developed processor system, resulting in increasingly prominent bottlenecks in storage technology. Among them, the bottleneck phenomenon of the development of storage technology is mainly reflected in two aspects: the writing and reading speed of data in the storage system is low. [0003] In the prior art, to increase the writing and reading speed of data in the storage system, data writing and reading can be performed in an all-memory manner. Specifically, after receiving the data write command, write the data to be written into the dynamic random access memory (Dynamic Random Access Memory, DRAM), so that after receiving the data read command, it can be directly read from the DRAM Read the corresp...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06G06F17/30
CPCG06F3/0689G06F3/0613G06F3/064
Inventor 金添福李明君任努努
Owner HUAWEI CLOUD COMPUTING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com