Tampered video detection method and system based on multi-domain block feature mark point registration

Abstract

The invention discloses a tampered video detection method and system based on multi-domain block feature mark point registration. The method comprises the following steps: extracting a to-be-detected area of each frame of image; constructing a feature extraction module; calculating an RGB image spatial domain feature and a DCT frequency spectrum of the to-be-detected region; extracting multi-scale spatial domain and frequency domain features and carrying out feature splicing to obtain fusion features; the fusion features are input into an attention module to generate attention guiding features; extracting local block features and key block features from the attention guiding features according to the face mark points; calculating the similarity between the block features and splicing the block features into a similarity matrix, and outputting a dichotomy prediction result; adopting a dichotomy label, a frame image local tampering probability and a frame image edge gradient to supervise network training; and performing prediction classification by using the trained model, and outputting a video tampering detection result. According to the method, the positions of the key block features are registered by using the face mark points, so that the generalization ability of the model is improved, and the learning of the model on the classification features of multiple fields is optimized.

Description

technical field [0001] The invention relates to the technical field of Deepfake video tampering detection, in particular to a tampering video detection method and system based on multi-domain block feature marker point registration. Background technique [0002] With the development of artificial intelligence, the technology of video tampering and forgery is also changing with each passing day. The Deepfake video tampering technology that has appeared in recent years uses a deep learning network to replace the original video face with the target face, while retaining the scene and character expressions of the original video. This technology has a low threshold for use and generates realistic video effects. If it is maliciously used and spread, it will have a negative impact on personal portrait rights and public opinion. Therefore, the research on Deepfake video tampering detection technology is of great significance. [0003] The existing Deepfake video tampering detectio...

AI Technical Summary

Problems solved by technology

Among them, the method based on manual feature extraction extracts manual features such as frequency domain map and optical flow map of video frame images, and inputs them into traditional pattern classifiers such as SVM for classification, which has the defect of insufficient feature extraction, resulting in poor detection results; The neural network feature extraction method inputs the video frame image into the neural network for feature mining, but does not consider the similarity relationship of different position features. The test in the library can achieve a better detection effect, but the cross-database detection performance is greatly reduced.

[0004] Although the above method has improved the in-database detection performance of face-changing video tampering detection to a certain extent, it has the problem of insufficient cross-database detection performance, which reduces the practicability and application value of the method

Images