Image restoration method based on cyclic feature reasoning of self-attention mechanism

Abstract

The invention discloses an image restoration method based on cyclic feature reasoning of a self-attention mechanism. The method comprises the following steps: constructing an SA-RFR network structure, wherein the SA-RFR network structure comprises a preprocessing module, a feature reasoning module and a self-adaptive ghosting fusion module; inputting the damaged image and the corresponding mask into a network structure, processing the damaged image and the corresponding mask through a preprocessing module, extracting a feature map, and judging and updating the mask; inputting the output of the preprocessing module into a feature reasoning module, and synthesizing partial content of the damaged image; alternately iterating the first two modules for six times, and storing the repaired feature map every time; and after the last repair is completed, combining all the feature maps generated in the repair process into a fixed feature map of one channel, and finally completing the repair through an adaptive ghosting fusion module to generate a final result. According to the technical scheme provided by the invention, the large-area missing image can be repaired, so that the repaired content details are clearer, the edges are smoother, and the structure is more reasonable.

Description

technical field [0001] The present application relates to the technical field of image inpainting, and more specifically, to an image inpainting method and system based on self-attention mechanism-based cyclic feature reasoning. Background technique [0002] In recent years, machine learning has developed rapidly, especially deep learning, which has become a popular direction in the computer industry in recent years. With the development of computer vision, many troubles that were difficult to solve in the past have good solutions today. For example, remove some specific objects in the image; people want to remove some unnecessary people or objects in the image after taking pictures; Many fields can play an important role, and with the rapid development of artificial intelligence, the application fields of image restoration will become more and more extensive. Therefore, the research on image inpainting based on deep learning is of great significance, and image inpainting ...

AI Technical Summary

Problems solved by technology

However, there are many disadvantages in the existing repair methods: First, the existing methods are almost all repairing the missing images in small areas. For images with too large missing areas, that is, the mask size is more than 50%, when repairing, usually Produce distorted structures or fuzzy textures that are inconsistent with surrounding areas; second, most methods for missing images in large irregular areas use generative confrontational network structures, which are computationally intensive and model training is prone to overfitting problems, making it difficult to implement network Training; third, in order to avoid the second problem, the end-to-end cyclic feature inference network structure RFR-NET is generally used to progressively repair missing images with large mask areas and irregular shapes through the shared feature inference module. However, RFR-NET also has some problems. When repairing the face dataset, it will generate unreasonable repair structures. When repairing the street view dataset, it will generate some similar diamond-shaped block textures, which seriously affect the repair effect.

Images