Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Model compression method and system and computer readable medium

A compression method and model technology, applied in the field of model compression, can solve the problems of not considering whether the model compression is suitable for the hardware environment, not considering the hardware environment, and the degree of automation is not high, so as to ensure the learning ability, the compression effect is good, and the volume is reduced.

Pending Publication Date: 2021-09-07
TONGJI UNIV
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The model compression method in the prior art needs to manually set the pruning rate of each layer, and manual intervention is required in the process of model compression, and the degree of automation is not high
In addition, the specific environment needs to be considered when compressing the model to determine the extent to which the model needs to be compressed. However, the existing model compression methods do not consider whether the model compression is suitable for the hardware environment. Although the compression effect is good, they do not consider Is it suitable for the current hardware environment, or is it overcompressed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model compression method and system and computer readable medium
  • Model compression method and system and computer readable medium
  • Model compression method and system and computer readable medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049]The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0050] A model compression method, the process of which is as follows figure 1 shown, including:

[0051] Step 1: Optimize and train the model on the training data set to obtain the original model after training, specifically:

[0052] Use the training data set to train the model, use the cross entropy between the model prediction label map and the real label map to optimize the original model, and obtain the original model after training;

[0053] Step 2...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a model compression method and system and a computer readable medium. The model compression method comprises the steps of 1, obtaining a trained original model; 2, searching the optimal pruning rate of each layer in the original model, and then performing filter-level pruning on the model; 3, quantifying the pruned model; and 4, performing knowledge migration in a knowledge distillation mode, and then performing fine tuning on the model after knowledge migration on the training data set to obtain a compression model. Compared with the prior art, the invention has the advantages of automatic compression, adaptability to hardware environment, small model size, good performance, convenience, rapidness and the like.

Description

technical field [0001] The present invention relates to the technical field of model compression, in particular to a model compression method, system and computer-readable medium based on neural architecture search. Background technique [0002] With the rapid development of deep learning technology in recent years, deep neural networks have shown excellent performance in many tasks, such as computer vision, speech recognition, natural language processing, etc., deep learning models rely on their powerful The learning ability has achieved the best performance on many mainstream data sets. However, deep learning models often have a large number of parameters, which take up a lot of computing resources during the training and inference stages, so they cannot be deployed on some resource-constrained devices, such as mobile phones, embedded devices, etc. . [0003] The goal of model compression is to achieve a model that is simplified from the original model without significan...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045
Inventor 赵生捷张斌张荣庆
Owner TONGJI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products