Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Quantification method and system based on neural network difference

A technology of neural network and quantization method, which is applied in the field of quantization method and system based on neural network difference, which can solve the problems of performance loss, large number of deep neural network model parameters, and high precision requirements, so as to maintain performance and achieve extremely low bit compression Effect

Active Publication Date: 2021-10-08
PEKING UNIV
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

On the other hand, the deep neural network model has a large number of parameters and high precision requirements, and the performance loss is relatively serious in the process of reducing the storage of the network model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Quantification method and system based on neural network difference
  • Quantification method and system based on neural network difference
  • Quantification method and system based on neural network difference

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0040] Such as figure 1 , image 3 As shown, the present invention provides a quantization method based on neural network difference, which specifically includes:

[0041] In the process of training the current network model, select the pre-trained network model and network structure of the relevant problem as the initialization of the current model;

[0042] In order to make full use of the pre-training model to reduce the size of the network model, the training parameter expression of the network model is used as the change amount based on the initialization model parameters;

[0043] Specifically, suppose the convolution parameters of the i-th layer of the pre-trained model are expressed as The convolution parameters of the newly trained model are denoted as W (i) . Under such assumptions, the operation of the current convolutional layer is expressed as:

[0044]

[0045] Among them, the network model L (i) Indicates the output of the i-th layer of the network mod...

Embodiment 2

[0063] The invention provides a quantization method based on neural network difference,

[0064] The network structure used is the SRCNN network model, where SRCNN is a three-layer convolutional neural network model. Specifically, the convolution kernel size of the first layer network model is 9×9, which is mainly used to extract the texture information on the input image; the convolution kernel size of the second layer network model is 1×1, which is mainly used to transform the input The characteristics of the image; the convolution kernel size of the last layer network model is 5×5, which is used to reconstruct the output image. After the first layer network model and the second layer network model, the ReLU activation function is added to increase the nonlinear transformation capability of the neural network.

[0065] The settings of the image quality restoration problem tested by the present invention are as follows. The images tested by the present invention are divided...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the field of digital signal processing, in particular to a quantization method and system based on neural network difference. Specifically include: training the network model, initializing the network model; using the training parameter expression form of the network model as the change amount based on the initialization model parameters; clustering and quantifying the change amount of the training parameter expression form of the network model, The corresponding compact representation of the network model is obtained. The invention solves how to improve the performance of the neural network model as much as possible under the condition of low-bit quantization under the condition that the pre-training model exists.

Description

technical field [0001] The invention relates to the field of digital signal processing, in particular to a quantization method and system based on neural network difference. Background technique [0002] With the continuous development of deep learning, it is more and more widely used in the fields of computer vision and natural language processing. It has been widely used in many problems such as image classification, image recognition and target detection, and image quality enhancement. On the other hand, with the increasing application of network models, the distribution and transmission of network models has gradually become an important research topic. In video coding and other related fields, in order to maximize the performance of the network model, relevant technical proposals point out that the transmission of the network model in the coded stream can obtain significant performance improvements, and the distribution and deployment of the network model is also invol...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06N3/04G06N3/08G06K9/62
CPCG06N3/08G06N3/045G06F18/23213
Inventor 王苫社赵政辉马思伟
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com