Edinburgh Research Explorer

Dual-modality Talking-metrics: 3D Visual-Audio Integrated Behaviometric Cues from Speakers

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Original languageEnglish
Title of host publication 2018 24th International Conference on Pattern Recognition (ICPR)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages6
ISBN (Electronic)978-1-5386-3788-3
ISBN (Print)978-1-5386-3789-0
Publication statusPublished - 29 Nov 2018
Event24th International Conference on Pattern Recognition - Beijing, China
Duration: 20 Aug 201824 Aug 2018

Publication series

ISSN (Print)1051-4651


Conference24th International Conference on Pattern Recognition
Abbreviated titleICPR 2018
Internet address


Face-based behaviometrics focus on dynamic biological signatures generated from face behaviors, which are informative and subject-specific for identity recognition. Most existing face behaviometrics rely on 2D visual features and thus are sensitive to pose or intensity variations. This paper presents a dual-modality behaviometrics algorithm (talking-metrics) that integrates 3D video and audio cues from a human face speaking a passphrase. Static and dynamic 3D face features are extracted algorithmically and audio features are transformed through a few learning models. We concatenate the top 18 discriminative 3D visual-audio features to represent the bi-modality and utilize an linear discrimant analysis (LDA) classifier for identity recognition. The experiments were conducted on a new publicly released dataset (S3DFM). Both qualitative feature distributions and quantitative comparison results show the feasibility of the proposed pipeline and the superiority over using each modality independently. A 98.5% cross-validation recognition rate over 60 subjects and 10 trials was achieved. An anti-spoofing test also demonstrates the robustness of the proposed method.


24th International Conference on Pattern Recognition


Beijing, China

Event: Conference

Download statistics

No data available

ID: 59198188