Image style migration system and method based on three-branch clustering semantic segmentation

Abstract

The invention discloses an image style migration system and method based on three-branch clustering semantic segmentation. The method comprises the steps of image preprocessing, image semantic segmentation, image content and style feature extraction, style matching and image similarity measurement. According to the method, the semantic segmentation technology is adopted, and the problem of style overflow possibly generated in the style migration process is effectively solved by applying the semantic segmentation technology to image style migration; the used MUNIT model belongs to an unsupervised deep learning model, and the model does not need a paired data set and can generate images of various styles, so the diversified requirements of users are met to a great extent; by adopting the step of the image similarity measurement algorithm based on the SSIM index, generation suppression of images with similar styles is realized, and the stability and effectiveness of the whole system are ensured while diversity requirements are met.

Description

technical field [0001] The invention belongs to the technical field of image processing and pattern recognition, and in particular relates to an image style migration system and method based on three-branch clustering semantic segmentation. Background technique [0002] There is a special type of application in the application of deep neural networks, that is, image style transfer. Image style transfer has been developed by Gatys, Johnson and others. Image stylization has been able to obtain satisfactory results under certain conditions. Currently popular image style transfer algorithms are mainly divided into two categories, one is slow style transfer based on image iteration, and the other is fast style transfer based on model iteration. Model iteration-based methods include feed-forward stylized models and GAN-based methods. There are two main representative works based on the feed-forward stylized model, namely the work of Johnson et al. and Ulyanov et al., while there ...

AI Technical Summary

Problems solved by technology

[0005] Although the above methods can achieve better style transfer effects, there are still the following problems: 1) The paired data sets are difficult to collect or even impossible to obtain, which brings great limitations to image style transfer; 2) After After training, often only one style of results can be obtained, which cannot meet the diverse needs of users; 3) only consider the similarity of the overall style of the image, and cannot retain the specific style of a specific object; 4) there is a problem of style overflow, which destroys the It improves the coordination and viewability of the whole image; 5) Other models that use clustering methods in the process of image style transfer in other technical solutions have the problem that the clustering effect is not good and affects the effect of style transfer6) There will be images with relatively high similarity among the multiple style images output by other technical solutions

Images