A complex visual image reconstruction method based on the dual model of deep codec

Abstract

The invention discloses a complex visual image reconstruction method based on a deep codec dual model, and belongs to the technical field of visual scene reconstruction in biomedical image brain decoding. The present invention first collects and watches functional magnetic resonance signals under a large number of natural images. Then establish four network models respectively: 1. Encoding model, which uses convolutional neural network to encode natural images into voxel signals in the visual area; 2. Decoding model, which uses convolutional neural network and deconvolutional neural network to convert The voxel signal in the visual area is decoded into a natural image; 3. Discrimination of the natural image model, that is, judging the real image and the fake image; 4. Discrimination of the visual area response model, that is, judging the real signal and the false signal. By training the designed four models, the visual scene image can be restored from the brain signal. The invention solves the problem of direct mutual conversion between natural scenes and brain signals for the first time, and can realize the practical application of brain-computer interface scenes.

Description

technical field [0001] The method belongs to the technical field of visual scene reconstruction in brain decoding of biomedical images, and specifically relates to the framework construction of a natural image reconstruction model of functional magnetic resonance images. Background technique [0002] In 2008, Miyawaki et al. asked subjects to watch a lot of flashing checkerboard stimulus pictures, and recorded the BOLD signal response of these stimuli in the early visual cortex (V1 / V2 / V3), using multi-voxel pattern analysis (Multi-voxel pattern classification, MVPA) method, established a multi-scale sparse multinomial logistic regression (SMLR) local decoder model, realized for the first time the brain signal visual image reconstruction not limited to the candidate visual stimulus category, and reconstructed simple geometric images and letters Stimuli, this study provides a new way of interpreting the visual perception state of the brain. However, the method of Miyawaki et ...

AI Technical Summary

Problems solved by technology

[0003] Aiming at the deficiencies of the background technology, the present invention solves the problems of low accuracy of the reconstructed image, relatively large noise, and relatively long reconstruction time, and improves and designs a dual model based on deep encoding and decoding on the basis of previous research. complex visual image reconstruction method

Images