Feature matching method based on attention mechanism and neighborhood consistency

Abstract

The invention provides a feature matching method based on an attention mechanism and neighborhood consistency. The method comprises the following steps: extracting a picture key point and a descriptor thereof by using a convolutional neural network; key point positions are coded, and position information is fused into descriptors; using a self-attention network and a cross-attention network to enhance the characterization capability of the descriptors and searching a corresponding relation between the descriptors of the two pictures; constructing a graph by using a k-nearest neighbor algorithm according to the attention weight, evaluating neighborhood consistency of nodes by using a graph neural network, and fusing consistency information into attention aggregation; and according to the enhanced descriptor, calculating a matching result by using a Sinkhorn algorithm and mutual neighbor screening. The invention provides a method for integrating neighborhood consistency into an attention mechanism, the neighborhood consistency in the attention aggregation process is dynamically evaluated while global context information is aggregated by the attention mechanism, and local information and global information are combined to jointly enhance the characterization capability of a descriptor, so that higher-quality matching is obtained.

Description

technical field [0001] The invention relates to a feature matching method and device technology based on attention mechanism and neighborhood consistency, and belongs to the technical field of computer vision based on deep learning. Background technique [0002] Feature matching refers to finding the correct correspondence from points and features of two images, and is a key link in many 3D computer vision tasks, such as structure from motion, simultaneous localization and mapping. Getting the right match makes a good foundation for these upstream tasks, but occlusion, blurring, similar repetitive textures, lighting and perspective changes can make it extremely challenging. Therefore, how to find the information hidden in the descriptors, such as context information and neighborhood consistency information, to eliminate interference is one of the most important processes in feature matching. [0003] The classic feature matching process generally consists of four parts: (1)...

AI Technical Summary

Problems solved by technology

This is an effective feature matching structure, but it will lead to the loss of neighborhood consistency when applying the attention mechanism to the feature matching task, because the attention network can be regarded as a graph neural network with all nodes fully connected

Images