Multi-modal medical image multi-organ positioning method based on one-to-one target query Transform

Abstract

The invention provides a multi-modal medical image multi-organ positioning method based on a one-to-one target query Transform, and belongs to the technical field of medical image processing. According to the method, the correlation between positions and sizes of organs is simulated by utilizing a conditional Gaussian model and a self-attention mechanism of Transform. The one-to-one target query architecture compulsorily executes a unique target query on each target organ, and the query sequence is the predicted category, so that classification is not needed, the network structure is simplified, redundant calculation is reduced, and the learning convergence speed is higher. According to the method, a 3D multi-modal image is projected to two orthogonal 2D planes before organ detection is executed, then complementary information from the multi-modal image is combined through a multi-modal fusion method, and finally, an obtained 2D bounding box is subjected to back projection to obtain a 3D bounding box, so that the calculation burden is reduced, and a more stable organ positioning result is obtained.

Description

technical field [0001] The invention belongs to the technical field of medical image processing, and specifically relates to a multi-modal medical image multi-organ positioning method based on a one-to-one target query Transformer, which mainly utilizes the geometric correlation between organs and the information complementarity of multi-modal medical images. Efficient and accurate automatic organ positioning. Background technique [0002] Multimodal medical images are common imaging modes in clinical medicine, such as Positron Emission Computed Tomography / Computed Tomography (PET / CT), T1, T2, and proton in MRI scans. Density maps, multiple spectral images in spectral CT, etc. Taking PET / CT images as an example, PET images are functional images that reflect the distribution of radioactive tracers in the body, and can be used for benign and malignant tumor diagnosis and tissue metabolism quantification; CT images reflect the absorption of X-rays by human organs and tissues ...

AI Technical Summary

Problems solved by technology

However, the Transformer network structure is complex and requires a lot of training time; and it needs a large amount of training data to perform well, but it is a time-consuming and laborious task for doctors to manually mark a large number of 3D medical images; the existing 3D Transformer is only through Reduce network depth to maintain a balance between accuracy and learnability

Furthermore, none of the current Transformer models use information from multimodal medical images to improve the accuracy of organ localization

Images