Face video heart rate estimation system and method based on token learning

Abstract

The invention discloses a face video heart rate estimation system and method based on token learning, and the method comprises the steps: 1, inputting a face video, and carrying out the face key point detection of each frame of the video; 2, acquiring a face region of interest by using the face key points, and extracting a multi-scale space-time diagram of the face video through color space conversion and pixel average pooling operation; 3, constructing a neural network model based on token learning to learn the multi-scale space-time diagram, and predicting an rPPG signal of the face video by using the trained neural network model based on token learning; and 4, carrying out peak point detection on the rPPG signal of the face video, and calculating a corresponding average heart rate value. According to the method, face video heart rate estimation is realized by using the neural network model based on token learning, so that the accuracy and robustness of face video heart rate estimation are improved.

Description

technical field [0001] The invention belongs to the field of physiological signal processing and relates to technologies such as computer vision, deep learning and signal processing, in particular to a face video heart rate estimation system and method based on token learning. Background technique [0002] Early heart rate estimation methods were mainly based on electrocardiographic techniques and contact photoplethysmography signals. These methods required specialized equipment to collect heart rate data by touching the patient's skin. However, the use of these contact sensors can be uncomfortable for the patient, especially for Those with sensitive skin, such as patients with skin burns and newborn babies, therefore, non-contact face video based heart rate estimation has attracted more and more attention. [0003] In recent years, heart rate measurement technology based on remote photoplethysmography (rPPG) signal has developed rapidly, which can obtain heart rate from fac...

AI Technical Summary

Problems solved by technology

However, this change in skin color is very subtle and is easily affected by various noises such as lighting and head movement

[0004] Early rPPG-based heart rate measurement methods mainly used color space conversion and signal decomposition methods to extract physiological signals from videos, and then these methods were limited by certain exact assumptions, such as specific skin reflection models and linear combinations, and could not be applied to other complex scenarios

With the rapid development of deep learning, some methods have begun to use deep learning with powerful modeling capabilities to solve the impact of various noises in face videos, but these methods mainly use traditional convolutional neural networks for video or manually extracted features. There are many drawbacks in image processing, especially the convolutional neural network is limited by the limited spatio-temporal receptive field, and often ignores the interaction between the long-range spatio-temporal receptive field, which leads to the low accuracy of the heart rate estimation result of the face video. Poor, and weak generalization, it is difficult to adapt to various complex real-world scenarios

Images