Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Processing unit, processor core, neural network training machine and method

A technology of neural network and processing unit, applied in the field of neural network, which can solve the problems of not saving computational overhead and memory access overhead

Pending Publication Date: 2021-06-22
ALIBABA GRP HLDG LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When implemented in software, the pruned weight gradient is still calculated, which does not actually save computational overhead and memory access overhead

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Processing unit, processor core, neural network training machine and method
  • Processing unit, processor core, neural network training machine and method
  • Processing unit, processor core, neural network training machine and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] The present invention is described below based on examples, but the present invention is not limited to these examples. In the following detailed description of the invention, some specific details are set forth in detail. The present invention can be fully understood by those skilled in the art without the description of these detailed parts. In order to avoid obscuring the essence of the present invention, well-known methods, procedures, and flow charts are not described in detail. Additionally, the drawings are not necessarily drawn to scale.

[0050] The following terms are used in this document.

[0051] Neural Network: Generally refers to Artificial Neural Network (ANN for short), which is an algorithmic mathematical model that imitates the behavior characteristics of animal neural networks and performs distributed parallel information processing. This kind of network depends on the complexity of the system, and achieves the purpose of processing information by...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a processing unit, a processor core, a neural network training machine and a method. The processing unit comprises a calculation unit used for executing weight gradient calculation of neural network nodes; the decompression unit decompresses the obtained compressed weight signal into a weight signal and a pruning signal, the weight signal indicates the weight of each neural network node, and the pruning signal indicates whether the weight of each neural network node is used in weight gradient calculation or not; the pruning signal is used to control whether to allow access to an operand memory storing an operand used in the weight calculation, and the pruning signal is also used to control whether to allow the calculation unit to perform a weight gradient calculation using the weight signal and the operand. According to the invention, the calculation overhead of a processor and the access overhead of a memory are reduced when the weight gradient of the neural network is determined.

Description

technical field [0001] The invention relates to the field of neural networks, and more specifically, to a processing unit, a processor core, a neural network training machine and a method. Background technique [0002] In the process of training the neural network (NN), it is necessary to repeatedly solve and update the weights of the neural network nodes. When solving the weights, the weight gradient needs to be determined first, and the gradient descent method is used to solve the weights based on the weight gradient and other operands. The calculation of the weight gradient accounts for a large part of the calculation and storage resources of the entire neural network. If the weight gradient is solved for each neural network node, it will take up a lot of resources. Therefore, pruning can be performed in the early stage of training, and some neural network nodes that have little influence on the calculation results are not considered when calculating the weight gradient....

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N3/08G06N3/04G06N3/063
CPCG06N3/082G06N3/063G06N3/045G06N3/084G06F9/30036
Inventor 关天婵高源柳春笙陈教彦
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com