Crowd counting model based on structured knowledge distillation and method thereof

Abstract

The invention discloses a crowd counting model and method based on structured knowledge distillation. The method comprises the steps: S1, obtaining a crowd image, and generating a corresponding real crowd density map through the marked information; s2, inputting different crowd images into a teacher network of a heavy magnitude for pre-training by multiple iterations, extracting features of each layer, and generating an estimated crowd density map; s3, inputting the crowd image into a lightweight student network, extracting features of each layer, and generating an estimated crowd density map;s4, calculating unary knowledge similarity and paired knowledge correlation coefficients of the corresponding student network and teacher network features; s5, calculating the loss through the unitary knowledge similarity, the paired knowledge correlation coefficients and the crowd density graphs, and updating the parameters of the student network; and S6, iteratively carrying out training processes of S1 and S3-S5 for multiple times by utilizing different crowd images until a stop condition is met.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a crowd counting model and method based on structured knowledge distillation. Background technique [0002] Crowd counting is an important technique in crowd analysis, the goal of which is to automatically calculate the total number of people in a surveillance situation. With the rapid increase of urban population, crowd counting has a wide range of applications in video surveillance, congestion warning, and traffic management, which makes it a hot research topic. [0003] In recent years, deep neural networks have become the mainstream method in the field of crowd counting, and have achieved remarkable progress in representative learning ability. In order to make better progress, some cutting-edge methods use heavyweight backbone networks (such as VGG) to extract features, for example, the research work of Yuhong Li et al. in 2018 "CSRNet: Dilated convolutional neural n...

AI Technical Summary

Problems solved by technology

In order to make better progress, some cutting-edge methods use heavyweight backbone networks (such as VGG) to extract features, for example, the research work of Yuhong Li et al. in 2018 "CSRNet: Dilated convolutional neural networks for understanding the highly congested scenes" (IEEEConference on Computer Vision and Pattern Recognition (CVPR), 2018) proposed a CSRNet network structure for understanding complex crowded scenes. The front end of the network uses the convolutional layer of VGG-16 to extract two-dimensional features, and the back end uses the expansion The convolutional layer expands the receptive field of feature extraction and has achieved remarkable results in crowded crowd scenes. However, this method is very inefficient, requires huge computing costs, and runs very slowly. Therefore, how to pass the existing pre-training but Clunky network to obtain efficient crowd counting becomes the current problem to be solved

[0004] For this problem, there have been many attempts to compress and accelerate convolutional neural networks, but these methods either require cumbersome hyperparameter adjustments, or rely on specific hardware platforms

[0005] Recently, the widely used knowledge distillation (Knowledge Distillation, or KD) method is a desirable choice, which imitates the knowledge of a complex heavyweight teacher network by training a simple lightweight student network, which has been proved by many works. Its effectiveness on image classification, but how to distill existing heavyweight models on crowd counting is an unexplored problem

Images