Performance data acquisition method for large-scale parallel program

Abstract

The invention discloses a load balancing based performance data acquisition method for a large-scale parallel program. The method is used for performing distributed acquisition on performance data generated by a large-scale parallel program in a high-performance computer and saving the data in an external memory. Nodes in the high-performance computer are divided into three classes of compute nodes, acquisition nodes and control nodes with the method comprising the following steps of: dynamically selecting the needed acquisition nodes with a load balancing algorithm; instrumenting and committing the parallel program on the control nodes through tools; collecting the performance data of the parallel program and sending the performance data to the selected acquisition nodes in a distributedway; receiving and caching the performance data from all the compute nodes in a distributed way and writing the performance data in the external memory. The efficiency and the expandability of the acquisition of the performance data of the parallel program can be improved with the method.

Description

technical field [0001] The invention can be applied to the field of large-scale parallel program data acquisition in high-performance computer systems. Background technique [0002] With the high-performance computing technology becoming more and more mature, parallel programs have been paid more and more attention to and used by more and more people. At present, the actual utilization rate of high-performance computers is very low. The main reason is that the task scheduling in parallel programs is not reasonable enough and the load is unbalanced, so that most processors are in an idle state. Without the necessary performance monitoring and visualization tools, it is difficult for programmers to monitor the execution status of parallel programs, communication status, processor utilization, etc., so that it cannot be accurately determined as the performance bottleneck of the algorithm. In this case, the development of parallel program performance evaluation tools has become...

AI Technical Summary

Problems solved by technology

The advantage of this data acquisition model is that no communication protocol is required and the implementation method is simple. The disadvantage is that the architecture is not scalable, and its acquisition efficiency will decrease significantly with the increase in the number of computing nodes.

If the parallel program runs on a supercomputer with a large number of nodes, the disadvantage of this data acquisition model will appear: the parallel program needs to write the performance data to the external memory while continuously generating performance data

If the number of computing nodes writing to the external memory is large, the conflict domain of computing nodes in file system operations will increase, and the efficiency of file system writing files will be greatly reduced, which may cause parallel programs to stop generating due to waiting to write files Performance data, which in turn affects the normal operation of parallel programs

Images