Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A depth neural network model clipping method based on statistical feature of feature graph

A deep neural network and statistical feature technology, applied in the field of pattern recognition and artificial intelligence, can solve problems such as being easily affected by bandwidth, low computing efficiency, and easy to destroy network parameter characteristics

Pending Publication Date: 2019-03-15
XIANGTAN UNIV
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] (3) The sparseness of the kernel. During the training process, the update of the weight is induced to make it more sparse. For the sparse matrix, a more compact storage method can be used, but the operation efficiency of the sparse matrix operation on the hardware platform is not high. High, easily affected by bandwidth, so the speedup is not obvious
[0008] a. The sparse method of the kernel only considers the compressed storage of the network, the compression effect is not obvious during operation, and the speed is not significantly improved;
[0009] b. Using the size of the weight as the evaluation index only considers the numerical characteristics of the weight itself, and does not consider the data characteristics of the network layer, so the compression effect is not high;
[0010] c. The calculation of the evaluation index is more complex and consumes more computing power;
[0011] d. The method of random cropping is too random, and it is easy to destroy the parameter characteristics of the network itself;

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A depth neural network model clipping method based on statistical feature of feature graph
  • A depth neural network model clipping method based on statistical feature of feature graph
  • A depth neural network model clipping method based on statistical feature of feature graph

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0055] The present invention will be described in detail below in conjunction with specific examples, and the following examples will help those skilled in the art to further understand the present invention. The examples described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0056] figure 1 It is a schematic flow chart of a deep neural network model clipping method based on feature map statistics in this example. By removing specific feature maps and corresponding convolution kernels, the network model framework can be reduced and the parameters can be compressed. The clipping method of the model, the specific implementation steps are:

[0057] (1) For the feature layer in the deep neural network, calculate the statistical features of each feature map in the feature layer in turn;

[0058] (2) Construct judging criteria based on statistical features;

[005...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a depth neural network model cutting method based on the statistical characteristics of a characteristic map. The method comprises the following steps: step 1, aiming at the characteristic layer in the depth neural network model, calculating the statistical characteristics of the characteristic map corresponding to each output channel; The characteristic layer consists of convolution layer and activation layer, or of convolution layer, normalization layer and activation layer. 2, accord to that statistical characteristics of the characteristic map correspond to each output channel in the characteristic layer, calculating evaluation indexes of each output channel in the characteristic lay; 3, judging that importance of each output channel in the characteristic layeraccord to the evaluation index, and removing the unimportant output channels and the corresponding parameter. The invention can effectively reduce the dimension of the neural network characteristic layer, improve the operation efficiency of the network model, reduce the network scale, and has little influence on the accuracy.

Description

technical field [0001] The invention belongs to the fields of artificial intelligence and pattern recognition, and in particular relates to deep neural network model compression. Background technique [0002] Deep learning (deep learning) has achieved remarkable results in solving advanced abstract cognitive problems, bringing artificial intelligence to a new level, and providing a technical basis for high-precision, multi-type target detection, recognition and tracking. However, complex calculations and huge resource requirements mean that neural networks can only be deployed on high-performance computing platforms, which limits their application on mobile devices. In 2015, Deep Compression, published by Han, applied network model clipping, weight sharing, quantization, and encoding to model compression, making model storage achieve good results, and also aroused researchers' research on network compression methods. The current research on deep learning model compression m...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G06F17/16
CPCG06F17/16G06N3/082G06N3/045
Inventor 周彦刘广毅王冬丽
Owner XIANGTAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products