Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Convolution kernel pruning model compression method and device

A compression method and convolution kernel technology, applied in the field of convolutional neural network model compression, can solve the problems of unimportant convolution kernel and insufficient clipping strength

Pending Publication Date: 2021-12-07
上饶市中科院云计算中心大数据研究院
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the traditional convolution kernel pruning process, there are two main problems. The first is the convolution kernel contribution evaluation criterion; currently, the more commonly used evaluation criteria include L1 norm, ApoZ, and methods based on absolute value. Each method has a certain theoretical basis and experimental proof, but there is no theoretically optimal and experimentally verified criterion.
Second: The pruning strength is not enough; in traditional convolution kernel pruning, the first n pruning units with the lowest importance are generally selected for pruning (n is determined by pruning sensitivity), but since there is no Unify the optimal importance evaluation criteria, so it will cause such a phenomenon: the pruning unit to be deleted is determined by pruning sensitivity, generally speaking, it is the least important, but the rest is considered important
In fact, due to the non-optimality of the importance evaluation criteria, this phenomenon is not necessarily correct, and the remaining convolution kernels after pruning may still be unimportant

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Convolution kernel pruning model compression method and device
  • Convolution kernel pruning model compression method and device
  • Convolution kernel pruning model compression method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

[0029] The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0030] A convolution kernel pruning model compression method, the method flow is as follows: the first step: for the use scene, use the commonly used model training method to train a convolutional neural network to convergence; the second step: for the convolution layer All feature maps calculate their correspo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a convolution kernel pruning model compression method and device, which are used in the field of convolutional neural networks, and the processing flow of the convolution kernel pruning model compression method comprises the following steps: firstly, for a use scene, training a convolutional neural network by using a common model training method until convergence; calculating two-dimensional image entropies corresponding to all feature maps in the convolution layer to form a convolution kernel importance array; then pre-cutting the convolution kernel, further observing the performance change of the model, and finally determining whether the convolution kernel is determined to be cut; after all convolutional layers in the model are traversed, ending the process; according to the improved convolution kernel pruning method, the convolution layer of a convolutional neural network is pruned, and a two-dimensional image entropy is put forward to be used as a criterion for importance evaluation of a convolution kernel; according to the method, on the basis of ensuring the performance of the convolutional neural network model, compared with traditional convolution kernel pruning, the convolutional layer is cut to a greater extent, and a greater degree of model compression effect is achieved.

Description

technical field [0001] The invention relates to a convolutional neural network model compression technology, in particular to a convolution kernel pruning model compression method and device. Background technique [0002] With the development of neural network, it has achieved excellent results in many aspects such as computer vision, target detection, natural language processing, etc., but the problems that follow are also obvious. The model size, parameter amount and calculation amount of deep neural network are all different Showing geometric growth, and due to problems in storage capacity, computing power and energy supply, etc., the deep neural network model cannot be directly deployed on resource-constrained devices such as mobile terminals and edge terminals, so now for deep neural network models The compression is of practical significance. The pruning method is a commonly used model compression method, including structured pruning methods and unstructured pruning m...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045
Inventor 马倩杨勇洪学海
Owner 上饶市中科院云计算中心大数据研究院
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products