Method, device and system for generating quantitative neural network, storage medium and application

A neural network and network technology, applied in the field of generating quantitative neural networks, can solve the problems of quantitative neural network performance impact, gradient information loss, gradient mismatch, etc.

Pending Publication Date: 2021-09-07
CANON KK
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the quantization process of the neural network, there is still the problem of gradient mismatch, that is, there is still the problem of loss of gradient information, so the performance of the generated quantized neural network will still be affected

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device and system for generating quantitative neural network, storage medium and application
  • Method, device and system for generating quantitative neural network, storage medium and application
  • Method, device and system for generating quantitative neural network, storage medium and application

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that the following description is merely illustrative and exemplary in nature and is in no way intended to limit the present disclosure and its application or uses. The relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in the embodiments do not limit the scope of the present disclosure unless specifically stated otherwise. Additionally, techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, they should be part of this specification.

[0027] Note that like numerals and letters refer to like items in the figures, so once an item is defined in one figure, it need not be discussed in the following figures. The present disclosure will be described in detail below with reference to the accompanying drawings.

[0...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method, device and system for generating a quantitative neural network, a storage medium and application. The method comprises the following steps: a determination step: based on floating point weights in a neural network to be quantized, respectively determining networks which correspond to the floating point weights and are used for directly outputting quantized weights; a quantization step: using the determined network to quantify the floating point weight corresponding to the determined network so as to obtain a quantized neural network; and an updating step of updating the determined network, the floating point weight and the quantization weight in the quantization neural network based on a loss function value obtained through the quantization neural network. According to the invention, the problem of gradient mismatching in a neural network quantization process can be solved, so that the performance of the generated quantization neural network can be improved.

Description

technical field [0001] The present invention relates to image processing, and in particular, to a method, apparatus, system, storage medium and application for generating a quantized neural network, for example. Background technique [0002] Currently, Deep Neural Networks (DNNs) are widely used for various tasks. With the increase of various parameters in the network, the resource load becomes a major problem in using DNNs for practical industrial applications. In order to reduce the storage resources and computing resources required for practical applications, quantizing neural networks has become a conventional method. [0003] In the process of quantizing a neural network (that is, in the process of generating a quantized neural network), since a large number of non-differentiable functions (for example, taking a sign operation (sign function)) are usually required, it will lead to gradient mismatch ( That is, the problem of gradient information loss), so that the perf...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045G06N3/08G06N3/04
Inventor 刘俊杰陈则玮温东超陶玮汪德宇
Owner CANON KK
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap