Naive Data Augmentation Might Be Toxic: Data-Prior Guided Self-Supervised Representation Learning for Micro-Gesture Recognition
Shah, Atif; Chen, Haoyu; Zhao, Guoying (2024-07-11)
IEEE
11.07.2024
A. Shah, H. Chen and G. Zhao, "Naive Data Augmentation Might Be Toxic: Data-Prior Guided Self-Supervised Representation Learning for Micro-Gesture Recognition," 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG), Istanbul, Turkiye, 2024, pp. 1-9, doi: 10.1109/FG59268.2024.10581907.
https://rightsstatements.org/vocab/InC/1.0/
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:oulu-202409205993
https://urn.fi/URN:NBN:fi:oulu-202409205993
Tiivistelmä
Abstract
Body gestures play an important role in nonverbal communication because they transmit emotional information. Recently, a specific group of gestures, so-called Micro-gestures (MGs), has drawn increasing research interests in the community, as they can be useful cues to interpret human inner feelings. In this study, we focused on recognizing MG via self-supervised learning from skeleton sequences with several contributions. Initially, we observed that existing data augmentation methods for skeleton data always fail in MG representation learning. Our investigation shows that the failure is caused by the inherent properties of real-world datasets, such as imbalanced/long-tail data distribution, intra-class ambiguity, and inter-class heterogeneity. Thus, we propose a novel prior-guided augmentation strategy that can preserve the original data distribution while maximizing the agreement between samples in self-supervised learning. Furthermore, we proposed a three-stream architecture of self-supervised presentation learning for micro-gestures via spatial/temporal masking to jointly enhance the learning of invariant features. Lastly, the experimental results show that our proposed method has achieved state-of-the-art performances on two public MG datasets.
Body gestures play an important role in nonverbal communication because they transmit emotional information. Recently, a specific group of gestures, so-called Micro-gestures (MGs), has drawn increasing research interests in the community, as they can be useful cues to interpret human inner feelings. In this study, we focused on recognizing MG via self-supervised learning from skeleton sequences with several contributions. Initially, we observed that existing data augmentation methods for skeleton data always fail in MG representation learning. Our investigation shows that the failure is caused by the inherent properties of real-world datasets, such as imbalanced/long-tail data distribution, intra-class ambiguity, and inter-class heterogeneity. Thus, we propose a novel prior-guided augmentation strategy that can preserve the original data distribution while maximizing the agreement between samples in self-supervised learning. Furthermore, we proposed a three-stream architecture of self-supervised presentation learning for micro-gestures via spatial/temporal masking to jointly enhance the learning of invariant features. Lastly, the experimental results show that our proposed method has achieved state-of-the-art performances on two public MG datasets.
Kokoelmat
- Avoin saatavuus [38824]