Grid data distributed storage service system, method, device, equipment and medium

Abstract

The present invention provides a grid data distributed storage service method, device, equipment and medium. The method includes: based on the principle of parallel subdivision, the grid data is dispersedly stored in the memory of each process, and the grid is constructed in a group Distributed storage of processes; according to the source parallel subdivision of the existing distributed storage on the grid, redistribute the grid data, and construct the distributed storage of the target parallel subdivision corresponding to the source parallel subdivision on the grid ; According to the distributed storage of the grid in a group of processes and the distributed storage of the target parallel subdivision, construct a distributed storage service model; according to the distributed storage service model, perform service functions; can effectively reduce network The running memory of grid data in a process improves the computing and processing capabilities of computers, and solves the performance bottleneck of storing large-scale grids in the development and application of numerical programs such as coupled models.

Description

technical field [0001] The present invention relates to the technical field of computer virtual storage technology, and in particular, to a grid distributed storage service system, method, apparatus, device and medium. Background technique [0002] Numerical programs used for simulation calculation usually need to discretely represent the simulated object as a grid with a certain dimension and resolution and the data on it. For example, the certain dimension can be one-dimensional, two-dimensional, three-dimensional, four-dimensional, or even More dimensions, the data includes grid data representing information such as the coordinates of each grid point, and data on each grid point used to characterize the properties of the simulated simulation object; and then numerically integrate the corresponding mathematical model on the grid to solve. Numerical programs usually require a large amount of computation. It is necessary to use MPI (message passing interface, message passing...

AI Technical Summary

Problems solved by technology

Although this global storage method greatly simplifies the program realization of functions such as online calculation of interpolation weights in the coupler, acquisition of information on required grid points, and output of grid information where coupling variables are located; it seriously increases memory requirements, especially When the model resolution is very high (a large number of grid points), it will cause the entire coupled model to run in parallel

For example, when the model resolution reaches 3 kilometers around the world, there will be more than 50 million grid points, and the memory usage for storing the entire horizontal grid information in one process can exceed 4GB, and there are usually dozens of processor cores but only a few memory points. The capacity usually does not exceed 64GB, which makes a computing node of the current high-performance computer can only run at most a dozen processes; when trying to run more processes, the computer system will forcibly interrupt the operation of the mode due to insufficient memory capacity, so it cannot Make full use of the computing power of high-performance computers

Images