ADL-GAN: Data Augmentation to Improve In-the-wild ADL Recognition using GANs

• Main Authors: Agu, E.O (Author), Ditthapron, A. (Author), Lammert, A.C (Author)
• Format: Article
• Language: English
• Published: Institute of Electrical and Electronics Engineers Inc. 2023
• Subjects: Activity of daily living; data augmentation; Data augmentation; GAN; Generative adversarial networks; Generators; imbalanced class; Legged locomotion; Sensors; Smart phones; smartphones; Task analysis; Training
• Online Access: View Fulltext in Publisher; View in Scopus
• Physical Description: 1
• ISBN: 21693536 (ISSN)
• DOI: 10.1109/ACCESS.2023.3271409

The types of Activities of Daily Living (ADL) a person performs or avoids, and underlying patterns can provide insights into physical and mental health, making passive ADL recognition from smartphone sensor data important. However, as people perform ADLs unequally in real life, ADL datasets collected in the wild can be extremely imbalanced, which presents a challenge to Machine Learning (ML) ADL classification. Prior solutions to mitigating imbalance, such as oversampling and instance weighting, reduce but do not completely eliminate the problem. We instead propose ADL-GAN, which utilizes translation Generative Adversarial Networks (GANs), to synthesize smartphone motion and audio sensor data to improve ADL classification performance. ADL-GANs augment the minority ADL of subject A by translating real samples from either 1) other ADLs where subject A has adequate data in <italic>Context-transfer ADL-GAN</italic> or 2) other subjects with adequate ADL data in <italic>Subject-transfer ADL-GAN</italic>. ADL-GANs utilize multi-domain and contrastive loss functions to perform many-to-many translations between ADL classes and subjects, respectively. Subject-transfer ADL-GAN outperformed baselines and improved balanced accuracy (BA) on an in-the-wild ADL dataset by 27.9 %, while context-transfer ADL-GAN performed best on a scripted dataset, improving the BA of baselines by 9.58 %. The augmented samples from ADL-GANs were shown to be more realistic and diverse than conditional GAN. Author