Multimodal feature selection for detecting mothers' depression in dyadic interactions with their adolescent offspring

Maneesh Bilalpur, Saurabh Hinduja, Laura A. Cariola, Lisa B. Sheeber, Nick Alien, Laszlo A. Jeni, Louis Philippe Morency, Jeffrey F. Cohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Depression is the most common psychological disorder, a leading cause of disability world-wide, and a major contributor to inter-generational transmission of psychopathol-ogy within families. To contribute to our understanding of depression within families and to inform modality selection and feature reduction, it is critical to identify interpretable features in developmentally appropriate contexts. Mothers with and without depression were studied. Depression was defined as history of treatment for depression and elevations in current or recent symptoms. We explored two multimodal feature selection strategies in dyadic interaction tasks of mothers with their adolescent children for depression detection. Modalities included face and head dynamics, facial action units, speech-related behavior, and verbal features. The initial feature space was vast and inter-correlated (collinear). To reduce dimension-ality and gain insight into the relative contribution of each modality and feature, we explored feature selection strategies using Variance Inflation Factor (VIF) and Shapley values. On an average collinearity correction through VIF resulted in about 4 times feature reduction across unimodal and multimodal features. Collinearity correction was also found to be an optimal intermediate step prior to Shapley analysis. Shapley feature selection following VIF yielded best performance. The top 15 features obtained through Shapley achieved 78 % accuracy. The most informative features came from all four modalities sampled, which supports the importance of multimodal feature selection.

Original languageEnglish
Title of host publication2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages8
ISBN (Electronic)9798350345445
DOIs
Publication statusPublished - 16 Feb 2023
Event17th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2023 - Waikoloa Beach, United States
Duration: 5 Jan 20238 Jan 2023

Publication series

Name2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023

Conference

Conference17th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2023
Country/TerritoryUnited States
CityWaikoloa Beach
Period5/01/238/01/23

Fingerprint

Dive into the research topics of 'Multimodal feature selection for detecting mothers' depression in dyadic interactions with their adolescent offspring'. Together they form a unique fingerprint.

Cite this