Local self-attention image processing method and model based on deformable block division

Abstract

The invention provides a local self-attention image processing method and model based on deformable block division. The method comprises the following steps: performing partitioning processing on a first feature map of an input image, predicting a first offset value of each block obtained by partitioning processing based on the first feature map, and correcting a range of each block obtained by partitioning processing based on the first offset value; performing feature extraction based on each corrected block in the first feature map to obtain a second feature map; performing windowing processing on the second feature map, predicting a second deviation value of each window obtained by the windowing processing based on the second feature map, and correcting a range of each window obtained by the windowing processing based on the second deviation value; and determining the self-attention of each block in the second feature map based on each corrected window, and taking the feature map after the self-attention is determined as a first feature map or a target feature map. According to the method, the performance of the model can be effectively improved under the condition of not increasing a large amount of calculation.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to an image processing method and model based on local self-attention of deformable blocks. Background technique [0002] Transformer (self-attention model) is a mainstream model in various tasks in the field of natural language processing, and has also attracted widespread attention in the field of computer vision in recent years. Compared with traditional convolutional networks, Transformer has obvious advantages in establishing long-distance relationship models and fitting large-scale data sets. At present, Transformer has achieved performance beyond the traditional convolution model in the fields of image classification, target detection, and semantic segmentation. [0003] Transformer mainly includes two modules, self-attention module and fully connected forward module. The earliest Transformer evenly divides the input image into several small area blocks according to...

AI Technical Summary

Problems solved by technology

However, this kind of crude windowing for self-attention calculation prevents some adjacent blocks from participating in self-attention calculation, even if they have a high similarity, which damages the representation ability of the model and reduces the performance of the model. performance

Images