Method, device and equipment for updating network model, and nonvolatile storage medium

A technology of a network model and a neural network model, applied to a method for updating a network model, devices and equipment, and the field of non-volatile storage media, which can solve problems such as low communication efficiency, improve throughput, reduce communication traffic, The effect of improving communication efficiency

Pending Publication Date: 2022-04-22
ALIBABA GRP HLDG LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present application provides a method, device and equipment for updating the network model, and a non-volatile storage medium, so as to at least solve the technical problem of low communication efficiency between artificial intelligence computing machines in the related art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device and equipment for updating network model, and nonvolatile storage medium
  • Method, device and equipment for updating network model, and nonvolatile storage medium
  • Method, device and equipment for updating network model, and nonvolatile storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0034] According to an embodiment of the present application, there is provided an embodiment of a method of updating the network model, it is emphasized that the steps shown in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer executable instructions, and, although a logical order is shown in the flowchart, but in some cases, the steps shown or described may be performed in an order different from herein.

[0035] Embodiments of the method provided in Example 1 of the present application may be performed in a mobile terminal, a computer terminal, or a similar computing device. Figure 1 Shows a hardware block diagram of a computer terminal (or mobile device) for implementing a method of updating the network model, such as Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b,......,102n shown in the figure) processor 102 (processor 102 may include, but is not limited to, a mi...

Embodiment 2

[0123] According to an embodiment of the present application, further provides an embodiment of an apparatus for implementing the above method of updating the network model, Figure 6 Is a schematic structural diagram of an apparatus for updating the network model according to an embodiment of the present application, such as Figure 6 As shown, the apparatus comprises: acquisition module 400, sampling module 402, selection module 404, estimation module 406 and update module 408, wherein:

[0124] Acquisition module 400, for obtaining the global gradient distribution of the deep neural network model, wherein the global gradient distribution comprises: the gradient distribution information of some or all layers of the above deep neural network model; sampling module 402, for sampling the gradient data generated by the above deep neural network model based on the distributed training scenario, to obtain gradient position information; select module 404, for using the above gradient pos...

Embodiment 3

[0128] According to an embodiment of the present application, there is also provided an embodiment of a device that updates the network model, the device that updates the network model may be any one of the computing devices in the computing device group. Figure 7 is a schematic structural diagram of an updated network model apparatus according to an embodiment of the present application, such as Figure 7 As shown, the updated network model of the apparatus comprises: processor 500 and memory 502, wherein:

[0129] Processor 500; and the memory 502, connected to the processor 500, for providing instructions for the processor to process the following processing steps: to obtain a global gradient distribution of the deep neural network model, wherein the global gradient distribution comprises: some or all of the layers of the above-described deep neural network model gradient distribution information; based on the above global gradient distribution in the distributed training scenar...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method, a device and equipment for updating a network model, and a nonvolatile storage medium. The method comprises the steps that global gradient distribution of a deep neural network model is acquired, and the global gradient distribution comprises gradient distribution information of a part of or all layers in the deep neural network model; sampling the deep neural network model in a distributed training scene based on the global gradient distribution to obtain gradient position information; selecting a plurality of target gradients by using the gradient position information; performing gradient estimation according to the plurality of target gradients to obtain a gradient estimation result; and updating the deep neural network model through the gradient estimation result. The technical problem that the communication efficiency between artificial intelligence computers is low in the prior art is solved.

Description

Technical field [0001] The present application relates to the field of computer technology, in particular, to a method of updating the network model, apparatus and apparatus, nonvolatile storage medium. Background [0002] With the continuous development and progress of artificial intelligence technology, the use of distributed training methods to train large data sets has become an important means of output of artificial intelligence computing platforms. In large-scale distributed training scenarios, the communication efficiency between artificial intelligence machines is low due to the limited constraint of communication bandwidth, and if the communication delay between artificial intelligence computers is too large, the overall training efficiency and multi-machine scalability of the training system cannot be guaranteed. [0003] Therefore, under the premise of ensuring that the accuracy is not reduced as much as possible, the communication efficiency between artificial intel...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08H04L41/147
CPCG06N3/08H04L41/145G06N3/045
Inventor 宋刘一汉潘攀刘宇张迎亚徐盈辉
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap