Brain map classification method based on deep multi-modal graph convolution

Abstract

The invention provides a brain image classification method based on deep multi-modal image convolution. The purpose of brain image classification is achieved by fusing brain images of different modals. The method comprises the following steps of: firstly, constructing a multi-modal brain topological graph, and constructing the brain topological graph by using resting state functional magnetic resonance data and diffusion tensor magnetic resonance data according to biological meanings of the resting state functional magnetic resonance data and the diffusion tensor magnetic resonance data; and then, carrying out multi-modal fusion, wherein the multi-modal fusion comprises a function-structure fusion part and a dynamic-static fusion part. According to the invention, multiple modal features are used and fused, and the similarity and complementarity among the features can be fully utilized, so that a brain map classification result is more accurate.

Description

technical field [0001] The invention relates to a brain map classification method for deep multi-modal map convolution, and belongs to the technical field of pattern recognition in computer image recognition. Background technique [0002] Mental illness has become a common health problem in the world today. It not only disrupts the lives of patients, but also has a huge impact on economic development and social stability. Therefore, the problem of mental illness diagnosis has received more and more attention from all walks of life. At present, mental illness is mainly diagnosed based on scale inquiries, doctor's consultation and clinical observation, which is easily affected by the large differences in clinical symptom groups of patients, the professional level of doctors and the subjective factors of patients, which brings challenges to accurate diagnosis. In recent years, electroencephalogram (Electroencephalogram, referred to as EEG), magnetoencephalography (Magnetoencep...

AI Technical Summary

Problems solved by technology

However, while multimodal features provide rich information for brain map classification problems, there is a problem of too high feature dimension.

Moreover, the biological significance represented by each modality is different, and the characteristics represented are inconsistent when distinguishing patients from normal people, resulting in feature heterogeneity between modalities

Feature heterogeneity and high-dimensional redundancy will affect the accuracy and stability of brain map classification, and the existing multimodal brain disease diagnosis methods are difficult to directly apply

Images