125 results about "Modal data" patented technology

The invention relates to a three-dimensional brain tumor image segmentation method based on an improved U-Net neural network. The method comprises the following steps: firstly, performing bias field effect removal processing of an N4ITK algorithm on three-dimensional brain tumor MRI image data, and secondly, respectively performing gray normalization preprocessing on each modal image in an original MRI image; building and training an improved U-Net convolutional neural network model; in a training process, inputting the four types of modal data of the patient as four channels of a neural network into an improved U-Net convolutional neural network model for training; the convolutional neural network U-Net being used as a basis, establishing an improved U-Net convolutional neural network model, and the improved U-Net convolutional neural network model comprising an analysis path used for extracting features and a synthesis path used for recovering a target object.

The invention discloses a multi-modal human face recognition method based on deep learning, and the method comprises the steps: (1), carrying out the human face detection and alignment of an RGB human face image; (2), designing a multi-modal fusion depth convolution neural network structure N1, and training an N1 network; (2), designing a multi-modal shared depth convolution neural network structure N2, and a training an N2 network; (4), extracting features; (5), calculating similarity; and (6), carrying out the similarity fusion. The method employs a multi-modal system, overcomes some inherent shortcomings of a single-modal system through the collection of various types of human face modal data, the advantages of various types of modal information and a fusion strategy, effectively improves the performance of a human face recognition system through the efficient use of various types of modal information, and enables the human face recognition to be quicker and more accurate.

The invention discloses a cross-modal hash retrieval method and system for fusing supervision information. The method comprises the following steps: constructing an image network, a text network and afusion network; The training sample pairs of image and text features are obtained and input into image network and text network respectively. Taking the output characteristics of the image network and the text network as the inputs of the fusion network, and defining the output of the fusion network; Constructing an objective function for learning a unified hash code according to the output of the converged network and the similarity between pairs; Solving the objective function to obtain a unified hash code; The unified hash code is used as the supervisory information and the semantic information is combined to train the hash network of a specific mode. The invention simultaneously learns the feature representation and the hash coding based on the end-to-end depth learning framework, canmore effectively capture the correlation between different modal data, and is conducive to the improvement of the cross-modal retrieval accuracy.

The invention discloses a mobile type multi-modal interaction method and device based on enhanced reality. The method comprises the following steps that: through an enhanced reality way, displaying ahuman-computer interaction interface, wherein an enhanced reality scene comprises interaction information, including a virtual object and the like; through the ways of gesture and voice, sending an interaction instruction by a user, comprehending different-modal semantic through a multi-modal fusion method, and carrying out fusion on the modal data of the gesture and the voice to generate a multi-modal fusion interaction instruction; and after a user interaction instruction acts, returning an acting result to an enhanced reality virtual scene, and carrying out information feedback through thechange of the scene. The device of the invention comprises a gesture sensor, a PC (Personal Computer), a microphone, optical transmission type enhanced reality display equipment and a WiFi (Wireless Fidelity) router. The invention provides the mobile type multi-modal interaction method and device based on the enhanced reality, a human-centered thought is embodied, the method and the device are natural and visual, learning load is lowered, and interaction efficiency is improved.

The invention discloses a cross-modal retrieval method based on a cyclic generation antagonistic network. The method designs a novel dual-channel cyclic generation antagonistic neural network and establishes semantic correlation of cross-modal data by training the neural network. Given different modal data can flow bi-directionally in the network, each modal data can generate another modal data through a group of generative antagonistic networks, and the generated data can be used as the input of the next group of generative antagonistic networks, so that the bi-directional loop generation ofdata can be realized, and the network continuously learns the semantic relationship among the cross-modal data. In order to improve the efficiency of retrieval, The method also uses threshold functionand approximation function to approximate the result of the middle layer of the generator to the corresponding binary hash code, and designs a variety of constraints to ensure the same mode, the similarity and cross-mode of the same kind of data, and the difference of data between classes, thus further improving the accuracy and stability of retrieval.

The invention relates to a cross-modal joint hash retrieval method based on self-supervision. The method comprises the following steps: step 1, processing image modal data: carrying out feature extraction on the image modal data by adopting a deep convolutional neural network, carrying out Hash learning on the image data, and setting the number of nodes of the last full connection layer of the deep convolutional neural network as the length of a Hash code; step 2, processing the text modal data; using a word bag model for modeling text data, a two-layer full-connection neural network is established for feature extraction of text modal data, wherein the input of the neural network is a word vector represented by the word bag model, and the length of data of a first full-connection layer node is the same as that of data of a second full-connection layer node and a Hash code; step 3, for the neural network of category label processing, extracting semantic features from the label data by adopting a self-supervised training mode; and step 4, minimizing the distance between the features extracted from the image and the text network and the semantic features of the label network, so thatthe Hash model of the image and the text network can more fully learn the semantic features among different modals.

The invention discloses a multimodal depth learning-based remote sensing image classification method. The method includes the following steps that: a multimodal remote sensing image data sample set is constructed, wherein the multimodal remote sensing image data sample set includes multimodal remote sensing images which are formed based on different imaging principles; data modal sensitive feature learning depth networks are constructed, and different modal data are adopted to train corresponding feature learning depth networks; an inter-modal feature correlation model is constructed, correlation sharing features are generated and trained; and a test sample set is inputted into a trained and finely-tuned multimodal depth network, so that precise classification of the remote sensing images can be realized. With the multimodal depth learning-based remote sensing image classification method adopted, multi-source remote sensing images are effectively utilized; inter-modal complementary and cooperation information is mined; a whole process is executed automatically; low classification precision caused by artificial input and subjective human factors is avoided; and classification precision is improved.

The invention discloses a cross-modal search method capable of directly measuring similarity of different modal data. The method includes the steps of firstly, feature extracting; secondly, model building and learning; thirdly, cross-media data search; fourthly, result evaluating. By the method compared with traditional cross-media search methods, similarity comparison of different modal data can be performed directly, for cross-modal search tasks, a user can submit texts, images, sounds and the like of optional modals so as to search required corresponding modal results, requirements of cross-media search are satisfied, and search intensions of a user can be achieved more directly. Compared with other cross-media search algorithms capable of directly measuring similarity of different modals, the method is high in noise interference resistance and expression capacity of loosely-related cross-modal data, and better search results can be achieved.

The present invention provides an environmental excitation data multiple-test-based Bayesian model correction method. The invention aims to eliminate the defects of a traditional method. With the method adopted, data obtained by multiple tests can be directly processed and analyzed; modal parameters obtained by the tests can be inputted by one step, and a model correction result is directly outputted. The method provided by the technical schemes of the invention can be used for solving the correction problem of actual test data-based finite element model. The method of the invention is divided into two stages. In the first stage, the acceleration data of a structure under environmental excitation which are acquired by the multiple tests are analyzed, so that the natural frequency and vibration mode of the structure which are measured by each test can be obtained; and the nondeterminacy of the modal parameters is calculated, and the modal parameters are represented by a covariance matrix. In the second stage, the optimal values of the model parameters of the finite element model requiring to be corrected are obtained through optimizing an objective function and based on the modal data parameters of the structure which are obtained based on the multiple tests and the covariance matrix thereof as well as the objective function built according to the Bayesian theory.

The invention discloses an abnormal driving detection model establishment method, and the method comprises the steps: obtaining a plurality of first samples in a normal driving state, and enabling each first sample to comprise first modal data, collected by a plurality of collectors, in the same normal driving time period; Inputting the first modal data of the first sample into a first abnormal driving state detection model, and outputting a first prediction error value corresponding to the first sample; Adjusting parameters of the first abnormal driving state detection model by using a back propagation algorithm according to the first prediction error values corresponding to the plurality of first samples to form a second abnormal driving detection model; And setting a second abnormal driving detection model to judge a prediction error threshold value of the abnormal driving state. The invention further discloses an abnormal driving detection model establishing device and a storage medium. According to the method, the detection model of the abnormal driving behavior can be established by integrating the multi-modal data, and the detection precision and robustness of the abnormal driving detection model are improved.

The embodiment of the invention provides a face recognition method and device based on face attribute perception loss, and an electronic device. The method comprises: carrying out the scale normalization processing of a face image on the RGB modal data and Depth modal data of a face to be recognized, and carrying out the channel stacking fusion of the normalized face RGB modal data and the normalized face Depth modal data; inputting the face RGB modal data and the face Depth modal data which are subjected to stacking fusion into a deep convolutional network, and extracting high-dimensional face features of a face to be identified; inputting the high-dimensional face features into a face attribute perception loss and classification loss combination layer to realize face attribute-based feature clustering; and based on the feature clustering, realizing recognition of the to-be-recognized face by calculating similarity scores with different face images in the face image library. Accordingto the embodiment of the invention, the feature expression of the face can be more effectively optimized, so that the face recognition can be more accurately carried out.

The invention relates to a cross-modal correlation learning method based on multi-granularity hierarchical network. The method comprises the following steps: 1, building a cross-modal database containing multiple modal types, dividing the data in the database into a training set, a validation set and a testing set, processing the different modal data in the database in a partitioning way and extracting all modal original modal data and feature vector of data after partitioning; 2, training a multi-granularity hierarchical structure by utilizing the original data and partitioned data and learning unified representation for different modal data; 3, obtaining the unified representation of different modal data by utilizing the trained multi-granularity hierarchical structure, and calculating the similarity of different modal data; 4, taking any modal type in the testing set as a query modal, taking another modal type as a target modal, calculating the similarity between a query sample and a query target and obtaining the related result list of the target modal data according to the similarity. According to the method of the invention, the accuracy rate of cross-modal searching can be improved.

The invention discloses a cross-modal retrieval method based on a deep correlation network. The method comprises the following steps that a primary vector of first modal data and a primary vector of second modal data are obtained through an initial feature extraction method; secondly, a deep correlation network model is established and trained, and high-level representation vectors corresponding to a retrieval target and retrieval library members are obtained through the deep correlation network model; thirdly, similarity matching is conducted on the retrieval target and each retrieval memberin a retrieval library through the high-level representation vectors, that is, the Euclidean distance is calculated; fourthly, calculation results of the Euclidean distance are successfully distributed from small to large, and a cross-modal retrieval result list of the retrieval target is obtained. According to the method, multiple layers of corresponding incidence relations are built between dataof different modals, and meanwhile various neural networks are fused, so that a deep model has a better representation effect; the precision of the cross-modal retrieval is higher, and the stabilityis better.

The invention discloses a finite element model correction method for a large-span steel bridge based on a uniform temperature response monitoring value, and the method comprises the following steps: 1), carrying out the analysis of yearly monitoring data of the large-span steel bridge, and finding a response law of a structure under the action of a uniform temperature field; 2), building a preliminary finite element model according to design data; 3), preliminarily determining the horizontal rigidness of a steel bridge supporting seat through employing an iteration method; 4), carrying out the sensitivity analysis of the large-span bridge based on the temperature response data of the large-span bridge, and determining a design variable with the higher coefficient of association with actual measurement data; 5), carrying out the optimization analysis of a large-span bridge finite element model through reducing the difference between a finite element model calculation result and the actual measurement data. Compared with a commonly used finite element model correction method based on a power response result (test modal data), the method is simple and accurate, is lower in cost, and is good in safety.

The invention discloses a topic modeling method based on a selected cell. The method includes extracting words, segment structures and word features contained in searching results in a database according to a query request; determining topics adopted by modeling; producing each segment structure topic, word topic and binary choice through random allocation; determining the variables through the Gibbs sampling process iteratively; feeding significant documents, words of each topic and capacities for words with various features to express the topic of the located segment structure to users according to final allocating results of the variables. The method has the advantages that topic modeling can be performed on various modal data; implicit structural information of the data is utilized fully, and disadvantages due to strong structural constraints are eliminated; information of correlation between the word features and the segment structural constraints can be provided, and the users are assisted to understand the data; the method has good extensibility and can serve as algorithm basis of various applications.

The invention discloses a multi-modal feature selection and classification method based on a hypergraph. The hypergraph can be used for effectively carrying out modeling on the high-order information of data. In the method disclosed by the invention, firstly, the hypergraph is independently constructed for the sample of each modal, and the Lapras matrix of the hypergraph is calculated; then, each group of modals is taken as one group of tasks, and a 12,1 norm is used for carrying out feature selection to guarantee that the characteristics of the same cerebral region of different modals can be selected. In addition, a hypergraph regularization term is used for describing high-order information between data samples so as to fully utilize a distribution prior in each group of modal data. Finally, a multi-core support vector machine is used for carrying out fusion classification on selected characteristics. By use of the method, the feature selection and classification can be effectively carried out.

The invention discloses a cross-media searching method based on discrimination correlation analysis. The method comprises the following steps of establishing a cross-media training database, carrying out feature extraction, mean-value pretreatment and linear projection transformation sequentially for different modal samples, and setting a target function according to a projection space; solving the target function to acquire a linear projection vector; establishing a cross-media test database; sequentially performing the feature extraction and mean-value pretreatment for a target to be searched; utilizing the linear projection vector to perform the linear projection transformation for the feature data after the mean-value pretreatment; and calculating an Euclidean distance between two modal data projection vectors, listing the Euclidean distance in ascending order, and acquiring a cross-media searching result. Due to the adoption of the method, dimensional reduction can be effectively performed for the feature data, so that the feature data can be widely applied to other multi-modal work, for example the multi-modal biological feature recognition.

The invention discloses a cross-media Hash index method based on a coupling differential dictionary. The cross-media Hash index method based on the coupling differential dictionary comprises the following steps that (1) modeling is conducted on the correlation of a plurality of modal data based on a graph structure, the similarity inside a modal is determined through the Euclidean distance between the data low-level features, the correlation between modals is determined by using the known correlation of different modal data, and classification label information of the data is used for improving the differentiation of the data on the graph structure; (2) the differential coupling dictionary is studied on the correlation of the data on the graph structure obtained in the step (1); (3) sparse coding is conducted on the different modal data by using the studied coupling dictionary in the step (2) and mapped inside unified dictionary space; (4) a Hash mapping function from the dictionary space to binary hamming space is studied. The cross-media Hash index method based on the coupling differential dictionary can realize the efficient cross-media searching of mass data based on content, and a user can submit a searching example of one modal to search a media object of another modal.

The present invention relates to a method and a system for an adaptive pattern recognition for psychosis risk modeling with at least the following steps and features: automatically generating a first risk quantification or classification system on the basis of brain images and data mining; automatically generating a second risk quantification or classification system on the basis of genomic and / or metabolomic information and data mining and further processing the first and second risk quantification or classification systems by data mining computing so as to create a meta-level risk quantification data to automatically quantify psychosis risk at the single-subject level. Preferably the first and / or second risk quantification or classification system(s) extract specific surrogate markers by multi-modal data acquisition and / or the surrogate markers are categorized and / or quantified by a multi-axial scoring system. Data can be controlled and outliers can be detected and eliminated preferably by determining cut-off thresholds. More preferably an outlier detection method transfers the brain image into a calibrated image, a segmented image and / or a registered image. Uni-modal data can be further generated and optionally optimized on the basis of the data acquired and one or more similarity and / or dissimilarity between the multi-modal data and the uni-modal data can be quantified.

The invention provides a data processing method and a data processing device and belongs to the technical field of mass data processing. The method comprises the following steps: acquiring an unstructured data resolving mode; resolving unstructured data according to the resolving mode so as to obtain modal data; calling a similar SQL (Structured Query Language) function class for processing pre-packaged unstructured data; processing the modal data according to the similar SQL function class. According to the method, the unstructured data are packaged into unified operators by virtue of classification of treatment; by virtue of designing similar SQL grammar, data processing is realized by virtue of calling in a similar SQL sentence, SQL data processing thought is applied to unstructured data processing, a package basic operation and a combination basic operation are adopted for realizing various complicated operation and reducing repeated operation, so that the complexity of realizing unstructured data processing is greatly reduced on the premise of not limiting the application range.

The present invention discloses a normal average value based bridge damage recognition method. The specific recognition is partitioned into eight steps of: selecting a bridge to be monitored, performing structure analysis on the bridge, and determining a monitoring item; performing layout of monitoring points, wherein layout contents comprise monitoring point codes, corresponding sensor types, and spatial three-dimensional coordinates; according to a layout scheme of the monitoring points, acquiring a stress, deformation, displacement, cable tension and modal data of each monitoring point for more than one year; according to an average value mu and standard deviation sigma of a statistic result, establishing a normal average value model of monitoring data corresponding to each monitoring point; and after establishing the normal average value model of different monitoring data of each monitoring point, according to position information and the spatial three-dimensional coordinates of each monitoring point, finding a damaged structure and a damaged position so as to facilitate bridge maintenance management personnel to perform maintenance and repair work, thereby ensuring security operation of a bridge. The normal average value based bridge damage recognition method provided by the present invention provides a corresponding determination condition for a bridge management and maintenance unit on bridge damage diagnosis and maintenance management, and has importance significance for development of a bridge health monitoring system.

The invention discloses a cross-modal searching method based on a topic model. The cross-modal searching method based on the topic model comprises the following steps that (1) feature extracting and label recording are conducted on all types of modal data in a database, (2) a cross-modal searching image model based on the topic model is built, (3) the cross-modal searching image model based on the topic model is solved by using a Gibbs sampling method, (4) a user submits data of a type of modal, and another relative type of modal data are returned by using the cross-modal searching model after feature extracting, and (5) the real corresponding information and the label information of the cross-modal data are used, and the cross-modal searching model is assessed from two aspects of correspondence and differentiation. The cross-modal searching method based on the topic model introduces the cross-modal topic and the different modal topic improving concept, the labor information is used, the interpretability and the flexibility of the topic modeling are improved and the cross-modal searching method based on the topic model has a good expandability and discrimination performance.

The invention discloses a data analysis method for multi-modal big data of clinical diseases. The data analysis method includes the steps: acquiring historical big data corresponding to disease types from a medical system; designing a multi-density quantizer according to change rates of the historical big data in various modes; extracting characteristic information corresponding to the disease types from the historical big data by a multi-modal data mining method and a convolutional neural network method; deducing the dynamic evolution law of individuals infected by the disease types in the historical big data according to the characteristic information; acquiring performance evaluation indexes of real-time data according to the dynamic evolution law of the individuals. Therefore, the method effectively overcomes limitation of single-modal data, dangerous factors of diseases can be taken into full consideration, more detailed and accurate clinical diagnosis standards are provided for hospitals, the dynamic evolution law of the individuals can be given, decision basis and technical support are provided for early diagnosis and early treatment of the diseases, and diagnosis efficiency and diagnosis and treatment quality are improved.

The invention discloses a nuclear power plant equipment monitoring system based on a wireless sensor network. A wireless monitoring node for gridding an equipment monitoring surface, obtaining tenth-order vibration mode inherent frequency and total displacement modal data of primarily selected monitoring points of important equipment to be monitored of the nuclear power plant, selecting optimized points by use of an intelligent optimization algorithm and collecting the monitoring information of nuclear power plant equipment monitoring terminals; a wireless relay node is used for gridding a spatial range in which the relay node is possibly deployed and primarily forming virtual redundant multi-route paths on the basis of the wireless monitoring node and further used for receiving the monitoring information data of the wireless monitoring node; a gateway and network access node is used for transmitting the monitoring information data to background monitoring software for processing; the background monitoring software is used for processing the monitoring information data and sending the processing result to a client and various monitoring terminals. The nuclear power plant equipment monitoring system based on the wireless sensor network is capable of meeting the special requirements such as high safety and high reliability in the application scenes of the nuclear power plant.

The invention discloses a unit temperature response monitoring value based correction method for a finite element model of a large-span steel bridge. The method comprises the following major steps of 1) analyzing annual monitoring data of the large-span steel bridge and determining static strain and displacement generated by unit uniform temperature change based on a relative probability histogram of a structure response value during unit temperature change; 2) establishing a primary finite element model according to design data; 3) preliminarily determining the horizontal stiffness of a steel bridge support by adopting an iterative method; 4) performing sensitivity analysis on the large-span steel bridge based on actual measurement data of displacement at the large-span steel bridge support and strain in a key position, and determining a design variable with a relatively high coefficient of correlation with the actual measurement data; and 5) performing optimization analysis on the finite element model of the large-span steel bridge by reducing a difference value of a finite element calculation result and the actual measurement data. Compared with a generally adopted finite element model correction method based on dynamic response results of test modal data and the like, the method has the advantages of simplicity, accuracy, relatively low expense and high security.

The invention requests to protect a multi-image regularization depth hash multi-modal medical image retrieval method. The method specifically comprises the following steps of: simultaneously extracting features of a multi-modal medical image group through a multi-channel depth model; Correspondingly constructing a plurality of graph regularization matrixes according to the characteristics of the multi-modal medical image group; fusing Multiple graph regularization matrixes, and obtaining Hash codes of the multi-mode medical image set through modal self-adaptive restricted Boltzmann machine learning; solving The distance between a single modal data hash code and a multi-modal medical image group hash code through Hamming distance measurement, carrying out sorting according to an ascending order, and selecting and returning n groups of multi-modal medical images with the minimum distance to a user, so that multi-modal medical image retrieval is realized. According to the method, a doctorcan be helped to quickly find data of other multiple modes through data of a certain mode in multi-mode medical images such as ultrasonic images, dispute end texts and nuclear magnetic resonance images, medical diagnosis of the doctor is facilitated, the workload of the doctor is reduced, and the working efficiency is improved.

The invention provides an online news multimodal emotion analysis method. The method disclosed by the invention comprehensively utilizes the text and image contents in the online news, constructs a multimodal depth learning model, realizes the full fusion of various modal data features, and analyzes and predicts the emotion of the reader after reading the news according to the multimodal depth learning model. For real news data with images, the method effect of the invention is significantly better than other emotion analysis models without considering image information, which proves that images in news are helpful to the elaboration of the whole news event and affect the reading emotion of readers.

The invention discloses an incomplete cross modal retrieval method based on subspace learning. The method comprises the steps of collecting multi-modal data and extracting features; establishing an incomplete observing multi-modal data set; learning shared subspace expressions of the incomplete observing multi-modal data set based on regression method; conducting feature learning on different modal data sets to select features with strong discrimination abilities; digging similarity relationships between incomplete observing modals and in the modals, and establishing an optimization objective function; obtaining the shared subspace expressions of multi-modal data and a projection matrix based on the regression method by the optimization objective function; conducting cross modal retrieval according to the projection matrix. The method solves the problem of modal feature heterogeneity, and at the same time can thoroughly utilize the data with complete modal and incomplete modal, and enhances the performance of the cross modal retrieval.