Convolutional neural network channel self-selection compression and acceleration method based on knowledge migration

Abstract

The invention discloses a convolutional neural network channel self-selection compression and acceleration method based on knowledge migration. The method comprises the following steps: S1, extractingmigration guidance knowledge from a trained complex convolutional neural network CN1; S2, setting randomly initialized coefficient masks for all convolutional layer channels of the target convolutional neural network ON1; S3, setting the output of each channel of the target network as a product of the original output of the channel and the corresponding coefficient mask; and S4, optimizing the target network under the guidance of the migration guidance knowledge, carrying out channel soft cutting according to the coefficient mask, and carrying out channel hard cutting until the target networkconverges. According to the method, the network can automatically select the channel to be cut off, manual selection operation is omitted, the expression capacity of the network is reserved and the generalization performance is improved through soft cut-off operation and a knowledge migration method, so that a higher compression ratio and a higher acceleration ratio are realized.

Description

technical field [0001] The invention relates to the technical field of deep learning and computer vision algorithms, in particular to a method for self-selecting compression and acceleration of convolutional neural network channels based on knowledge transfer. Background technique [0002] In recent years, the development of artificial intelligence has attracted more and more attention from all walks of life, especially in the image field such as computer vision, which has attracted the favor of scholars and industrial producers. Driven by the wave of artificial intelligence, visual recognition tasks such as face recognition and target positioning have achieved remarkable results, and the achievement of these results is inseparable from the innovation and development of deep convolutional neural network algorithms. However, the existing deep convolutional neural network models are very large in terms of storage and calculation, which hinders the application of this model on ...

AI Technical Summary

Problems solved by technology

However, the pruning of the network is carried out on the already trained network, and it will not be restored after pruning, so the problem of capacity reduction still exists. Therefore, it is studied how to effectively combine the two types of methods more scientifically to achieve More efficient compression and acceleration effects are necessary

Images