Deep neural network image recognition method based on structured natural gradient optimization

A deep neural network and natural gradient technology, applied in the field of deep neural network image recognition, to achieve the effect of excellent recognition performance and fast convergence speed

Pending Publication Date: 2022-04-22
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the technical problem that natural gradient descent is difficult to apply in the image recognition model based on deep neural network in image recognition, and creatively propose a deep neural network image recognition method based on structured natural gradient optimization

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep neural network image recognition method based on structured natural gradient optimization
  • Deep neural network image recognition method based on structured natural gradient optimization
  • Deep neural network image recognition method based on structured natural gradient optimization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0051] by figure 1 The MLP network shown is taken as an example, which can be optimized based on the structured natural gradient descent method based on the above-mentioned hierarchical structured natural gradient optimization idea. Such as figure 2 As shown, where f is an activation function, such as a linear rectification function (ReLU), an S-shaped function (Sigmoid), and the like. Taking the Sigmoid activation function as an example, for each neuron of the Sigmoid activation function, there are:

[0052]

[0053]Among them, v Sigmoid Represents the derivative of the Sigmoid activation function, Represents the transpose of the parameter vector w. x represents the input of the network layer.

[0054] The optimization method is as follows:

[0055] initialization They are all identity matrices.

[0056] Step A: Calculate (* represents an intermediate symbol representation operation on the original matrix), means G 1 the negative square root of the matrix, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a deep neural network image recognition method based on structured natural gradient optimization, and belongs to the field of image recognition based on machine learning and neural network technologies. According to the method, a structured natural gradient descent (SNGD) method is provided, a normalization layer is added, a deep network mode of restructuring image recognition is adopted, the correlation calculation of a global Fisher matrix is decomposed, and finally, the decomposition is converted into optimization by using traditional GD, so that the NGD effect can be achieved. Meanwhile, the invention provides a new local Fisher layer and an implementation scheme thereof. Second-order information is introduced into the local Fisher layer, different attributes of parameters in an image recognition network at different positions are considered, constraints are added to recognition model parameter transformation, and gradient updating can be conducted stably and rapidly. By adopting the method, the image recognition network training has higher convergence speed, and the trained model also has better recognition performance.

Description

technical field [0001] The invention relates to an image recognition method, in particular to a deep neural network image recognition method based on structured natural gradient optimization, and belongs to the field of image recognition based on machine learning and neural network technology. Background technique [0002] In the field of image recognition, how to train the deep neural network model stably and quickly is one of the main challenges faced in the application of deep network-based image recognition. At present, although the commonly used first-order traditional gradient descent (GD) optimization method is simple and effective, it often encounters the problem of ill-conditioned curvature (optimizing the canyon area on the surface). At this time, GD keeps bouncing along the ridge of the canyon, making it difficult to pass through Canyons, resulting in slower convergence during training. [0003] Natural Gradient Descent (NGD for short) is a more powerful optimiza...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06K9/62G06N3/04G06V10/764
CPCG06N3/082G06N3/045G06F18/241
Inventor 刘伟华刘峡壁李慧玉
Owner BEIJING INSTITUTE OF TECHNOLOGYGY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap