Multi-source data organization management method and apparatus

Abstract

The invention provides a multi-source data organization management method and apparatus. The method comprises the steps of S1, based on a management demand on target multi-source data, determining a design goal and a design principle of a target multilevel spatial data grid system; S2, based on the design goal and the design principle, establishing the target multilevel spatial data grid system byadopting a spatial reference of a universal transverse Mercator projection grid system; and S3, based on spatial feature information of the target multi-source data, correspondingly performing grid division and recombination on the target multi-source data according to the target multilevel spatial data grid system. The unity and normativity of multi-source data management can be effectively improved; the shareability and availability of the multi-source data are enhanced; and the complexity of an algorithm process can be effectively lowered.

Description

technical field [0001] The present invention relates to the technical field of data processing, and more specifically, to a method and device for organizing and managing multi-source data. Background technique [0002] The research process of large data storage environment and efficient computing mode of remote sensing data shows that distributed file systems (such as Hadoop Distributed File System (Hadoop Distributed File System, HDFS)) and cluster parallel computing (such as MapReduce mode (MapReduce)) Both embody the "divide and conquer" concept to a certain extent: in terms of data storage, the file data is split into several data blocks and stored in different data nodes to reduce the load of the data transmission network and improve the efficiency of data retrieval; in terms of data calculation and analysis , perform parallel calculations on different block data and aggregate the calculation results to improve calculation efficiency. [0003] Numerous related studies ...

AI Technical Summary

Problems solved by technology

[0005] (1) The grid division based on regular polyhedrons has complex algorithms, difficult coordinate conversion, and cannot guarantee complete equal-angle or equal-area characteristics;

[0006] (2) The adaptive grid division based on the spherical Voronoi diagram, the division rules are non-recursive, and the spatial entity objects and entity relationships all use the explicitly defined hierarchical grid, so the transformation operation of the spatial entity at a specific level cannot be transmitted to the adjacent It is also difficult to associate and interoperate with multi-scale massive data;

[0007] (3) Grid division based on spherical longitude and latitude lines. On the same scale, the area and shape of the grid are affected by the sphere, and the low-latitude and high-latitude areas are quite different. It is difficult to maintain the equal area of ​​the grid expression

Although there are related studies in different latitudes, based on different latitude and longitude differences, the grid area is approximately equal at the same scale, but the complexity of the grid organization is increased, and the shape deformation in the direction of latitude and longitude cannot be improved.

[0008] The above technical problems will adversely affect the organization and management of multi-source data, resulting in the loss of data in the organization and management of multi-source data, resulting in difficulties in data organization and retrieval

Images