Multi-mode three-dimensional medical image fusion method and system and electronic equipment

Abstract

The invention relates to a multi-mode three-dimensional medical image fusion method and system and electronic equipment. The method comprises the steps of constructing a multi-task generative adversarial network, wherein the multi-task generative adversarial network comprises a generator, a discriminator and a classifier; training the multi-task generative adversarial network according to the MRIimage, the PET image and the diagnosis label information of the subject; inputting an MRI image of a to-be-detected person into the trained multi-task generative adversarial network; and enabling thegenerator to synthesize a corresponding PET image according to the MRI image, input the MRI image of the to-be-detected person and the synthesized PET image into the classifier, and output a disease classification prediction label of the to-be-detected person after fusing the MRI image of the to-be-detected person and the synthesized PET image. According to the method, the conflict problem of lossfunction convergence points possibly occurring when the performance of a generator and the performance of a classifier are considered in a traditional generative adversarial network is solved, and the generator and the classifier can be optimal at the same time.

Description

technical field [0001] The present application belongs to the technical field of medical image processing, and in particular relates to a multimodal three-dimensional medical image fusion method, system and electronic equipment. Background technique [0002] Different from MRI (Magnetic Resonance Imaging, Magnetic Resonance Imaging), CT (Computed Tomography, Electronic Computer Tomography) and other images, PET (Positron Emission Computed Tomography, Positron Emission Computed Tomography) is a kind of human metabolic process. Functional imaging technology for in vivo observation has gradually been widely used in clinical diagnosis and early intervention. Specifically, the PET system can detect gamma rays emitted indirectly from radioactive tracers. First, the tracers are injected into the human body through bioactive molecules, and then computer analysis techniques are used to construct a three-dimensional PET image of the tracer concentration in the human body. The collect...

AI Technical Summary

Problems solved by technology

This will cause conflicts between sample generation and pattern classification convergence points. In other words, traditional generative models cannot be used to deal with multi-task problems.

At the same time, the existing cross-modal image synthesis method based on conditional generative adversarial network uses the image of a given modality as the conditional constraint information, without considering the label information of the sample

In addition, the existing cross-modal image synthesis and classification diagnosis research is to design independent models for the two tasks and train them separately, without considering the correlation between the two tasks in the optimization process

Images