Intelligent compression storage method and system for neural network check point data

Abstract

The invention provides an intelligent compression storage method for neural network check point data. The neural network check point data comprises weight floating-point number data and optimizer floating-point number data. The method specifically comprises the following steps: compressing and storing weight floating-point number data after each round of training of a neural network by using an incremental compression method; and / or utilizing index value mapping to replace the first n bits of part or all optimizer floating-point number data after each round of training of the neural network and then storing, wherein the number of bits of the index value is smaller than n. The intelligent compression method and system are designed by utilizing the characteristics of deep learning check point data and combining a model training process, and different types of data of the check points are efficiently compressed, so that the storage capacity of a storage system is effectively improved, and the service life of the storage system is effectively prolonged.

Description

technical field [0001] The invention relates to the field of computer science and artificial intelligence, in particular to an intelligent compression method and system for neural network checkpoint data. Background technique [0002] The innovation of deep learning technology has greatly promoted the development of computer vision, natural language processing, medicine and other fields, and has received great attention in both academia and industry. In order to obtain high accuracy, deep learning models need to be trained for a long time for iterative updates. During the model training process, a large number of parameters will be generated (some models can reach the data volume of GB or even TB level), and once the training process crashes, the task needs to be restarted. In order to prevent data loss, the parameters trained by the deep learning model need to be periodically stored in high-speed persistent devices (including SSD and non-volatile memory) in the form of che...

AI Technical Summary

Problems solved by technology

[0003] In order to solve the problem of insufficient device life caused by a large number of checkpoint write operations, the present invention proposes an intelligent compression method and system for checkpoint data, which compresses different types of floating-point data in checkpoint files by designing different mechanisms , to reduce the amount of data written to the persistent device and alleviate the life loss of the device

Images