RS+rPPG: Robust Strongly Self-Supervised Learning for rPPG

Abstract

Abstract

Remote photoplethysmography (rPPG) uses RGB facial videos to measure cardiac signals. It holds promise for future applications in telemedicine, affective computing, liveness-based face anti-spoofing, driver monitoring, etc. Supervised deep learning methods have been leading in performance but are severely limited by data availability, as recording face videos with ground truth physiological signals is expensive. Recent self-supervised methods aim to solve the data issue but struggle to learn robust features from data in challenging scenarios. These scenarios are characterized by overwhelming environmental noise caused by head movements, illumination variations, and recording device changes. We propose RS+rPPG, a novel contrastive method that effectively leverages a large set of eleven rPPG priors, enabling strong self-supervision even with challenging data. RS+rPPG comprehensively exploits intra-data and interdata information present in videos via diverse augmentations and learning constraints. We extensively experimented on seven rPPG datasets and demonstrated that RS+rPPG can outperform state-of-the-art supervised methods without using any labels. Additionally, we demonstrate the high generalization capability, demographic fairness, and mixed-data stability of our method.