Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data analysis and processing method and system based on read-ahead and slow-write

A technology for data analysis and processing and slow writing, applied in the direction of input/output to the record carrier, etc., can solve the problem of storage blockage and so on

Active Publication Date: 2018-04-17
BEIJING NORMAL UNIVERSITY
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The technical problem to be solved by the present invention is how to effectively reduce unnecessary waiting time during batch data calculation, improve program operation efficiency, and avoid storage blockage during batch input and output of data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data analysis and processing method and system based on read-ahead and slow-write
  • Data analysis and processing method and system based on read-ahead and slow-write
  • Data analysis and processing method and system based on read-ahead and slow-write

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The present invention will be further described in detail below with reference to the drawings and embodiments. The following examples are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0040] The invention discloses a data analysis and processing method based on pre-reading and slow-writing. The method includes:

[0041] S1. Read in some data to be input;

[0042] S2. Use a corresponding calculation method to calculate according to the to-be-input data read in step S1 to obtain a corresponding calculation result, and output the calculation result;

[0043] After the step S1 is executed for the first time, the steps S1 and S2 are executed at the same time, until all the data to be input are read into the step S1 and the execution stops; the step S2 stops until all the calculation results are output. carried out.

[0044] The above method embodies a read-ahead and slow-write mechanism: first, read in a part of the data...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data analysis and processing method and system based on pre-reading and slow writing. In the invention, a part of data to be input is first read in; then calculation is performed in advance according to the input data, and the remaining part or all of the data to be input is read in at the same time. Data, continue to calculate; at the same time, when there are calculation results that can be output, this part of the results will be output first. This processing method uses multi-threaded programming to run in parallel to complete the program in the shortest time. Dynamic and efficient use of memory; the introduction of this read-ahead and slow-write mechanism can increase program operating efficiency by up to 75%, and avoid storage blockage during batch input and output of data.

Description

Technical field [0001] The invention belongs to the field of high-performance remote sensing data processing, and more specifically relates to a data analysis and processing method and system based on pre-reading and slow-writing. Background technique [0002] In the process of using high-performance clusters for batch data calculations, data input and output operations frequently occur. Due to the large amount of calculation, multiple computing nodes access the memory at the same time, which will cause storage congestion and fail to achieve the ideal program. operating efficiency. The production procedures involved in the prior art are generally serial procedures, which basically go through the three processes of input, calculation and output. These three processes can be understood in the following three modules: [0003] Input module, when running the input module, input all data needed for calculation and temporarily store it in the memory, but it will take up a lot of space; ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06
Inventor 赵祥崔娇徐帅魏红梁顺林
Owner BEIJING NORMAL UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com