Neural network pruning method based on combination of sparse learning and genetic algorithm

Abstract

The invention discloses a neural network pruning method based on combination of sparse learning and a genetic algorithm, and mainly solves the problem that a neural network consumes a large amount ofstorage resources and computing resources. The implementation scheme is as follows; training a neural network in a sparse learning mode to obtain a sparse network structure; carrying out heuristic search on potential sub-networks in the trained neural network by utilizing a genetic algorithm and a dynamically adjustable evaluation factor, and automatically searching an optimal sub-network meetingrequirements under the guidance of a fitness function; and retraining the obtained optimal sub-network to obtain a final pruning result. According to the method, the consumption of storage resources and computing resources by the neural network is reduced, the precision of the pruned network is ensured, adjustable pruning can be realized, and the method can be used for compressing the neural network.

Description

technical field [0001] The invention belongs to the technical field of computers, and mainly relates to a neural network pruning method, which can be used for neural network compression. Background technique [0002] With the development of deep learning, neural networks have achieved good results in scientific research and industrial applications. However, compared with traditional algorithms, the implementation of neural networks requires a large amount of storage and computing resources. Therefore, neural networks are used on devices. It will generate a large amount of energy consumption, which is not in line with the concept of energy saving, and also limits the use of neural networks on mobile devices with limited power consumption. As a method of compressing the neural network, neural network pruning reduces the storage consumption and computing consumption of the neural network by removing redundant components in the neural network, so as to achieve the purpose of red...

AI Technical Summary

Problems solved by technology

[0004] Unstructured pruning is to compress the neural network by removing non-important weights in the neural network, but the positions of the removed weights in this type of method are out of order, thus destroying the original data structure of the neural network. Make the pruning result unable to obtain actual effect on the current general-purpose equipment

[0005] The structured pruning method is to achieve the purpose of compressing the neural network by removing non-important channels in the neural network. Since the removal of channels will not destroy the data structure of the neural network, this type of method can be well applied to existing computing devices. , but the structured pruning method is not as accurate as the unstructured method for the location of redundant components in the neural network, so this type of method has a low compression rate for the neural network and has a greater impact on the performance of the neural network

Images