3D SIFT (3D Scale-invariant feature transform) framework-based detection method and device of spatio-temporal interest points (STIPs)

Abstract

The invention discloses a 3D SIFT (3D Scale-invariant feature transform) framework-based detection method and device of spatio-temporal interest points (STIPs). The method and device combine gray information and motion change information. Compared with the prior art, the method constructs a GMCM (Gray and Motion Change Information Matrix) which is under a geometric algebra framework and contains the gray information and the motion change information, and constructs and obtains scale space through the GMCM; because the GMCM contains the gray information of pixels of a video in three-dimensional geometric algebraic space, and also contains the motion change information, the constructed and obtained scale space is related to the motion change information; and the scale space and a pyramid model constructed and obtained by utilizing the GMCM realize detection of the spatio-temporal interest points, the motion change information of the video on a time domain is fully utilized, and the repeatability, the robustness and other performance of the spatio-temporal interest points are enhanced.

Description

technical field [0001] The present invention relates to the field of video image processing, in particular to a method for detecting spatiotemporal interest points based on a 3D scale-invariant feature transform (3DScale-invariant feature transform, 3D SITF) framework. Background technique [0002] Behavior recognition in video is a hot research topic at present. Although the behavior recognition method based on deep learning has made breakthroughs in behavior recognition on large video data sets, it is difficult to apply it to small sample data sets. For example, the video data of traffic accidents, because the data of traffic accidents is difficult to collect and simulate, and the amount of data is very limited, it is difficult to apply deep learning methods to the identification of traffic accidents in videos. However, the traditional behavior recognition algorithms based on video spatio-temporal interest points (Spatio-Temporal Interest Point, STIP for short) and support...

AI Technical Summary

Problems solved by technology

[0005] The main purpose of the present invention is to provide a method and device for detecting spatio-temporal interest points based on 3D SIFT framework, aiming to solve the problem of insufficient use of motion change information in the time domain of video in the prior art and reduce the impact of spatio-temporal interest points of video. Technical issues of performance such as repeatability and robustness

Images