Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

Guosheng Hu, Hua Yang, Yang Yuan, Zhihong Zhang, Zheng Lu, Sankha S. Mukherjee, Timothy Hospedales, Neil M Robertson, Yongxin Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep learning has achieved great success in face recognition, however deep-learned features still have limited invariance to strong intra-personal variations such as large pose changes. It is observed that some facial attributes (e.g. eyebrow thickness, gender) are robust to such variations. We present the first work to systematically explore how the fusion of face recognition features (FRF) and facial attribute features (FAF) can enhance face recognition performance in various challenging scenarios. Despite the promise of FAF, we find that in practice existing fusion methods fail to leverage FAF to boost face recognition performance in some challenging scenarios. Thus, we develop a powerful tensor-based framework which formulates feature fusion as a tensor optimisation problem. It is nontrivial to directly optimise this tensor due to the large number of parameters to optimise. To solve this problem, we establish a theoretical equivalence between low-rank tensor optimisation and a two-stream gated neural network. This equivalence allows tractable learning using standard neural network optimisation tools, leading to accurate and stable optimisation. Experimental results show the fused feature works better than individual features, thus proving for the first time that facial attributes aid face recognition.
We achieve state-of-the-art performance on three popular databases: MultiPIE (cross pose, lighting and expression), CASIA NIR-VIS2.0 (cross-modality environment) and LFW (uncontrolled environment).
Original languageEnglish
Title of host publicationThe International Conference on Computer Vision (ICCV 2017)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages3764-3773
Number of pages10
ISBN (Electronic)978-1-5386-1032-9
ISBN (Print)978-1-5386-1033-6
DOIs
Publication statusPublished - 25 Dec 2017
Event2017 IEEE International Conference on Computer Vision - Venice, Italy
Duration: 22 Oct 201729 Oct 2017
http://iccv2017.thecvf.com/

Publication series

Name
PublisherIEEE
ISSN (Electronic)2380-7504

Conference

Conference2017 IEEE International Conference on Computer Vision
Abbreviated titleICCV 2017
CountryItaly
CityVenice
Period22/10/1729/10/17
Internet address

Fingerprint Dive into the research topics of 'Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks'. Together they form a unique fingerprint.

Cite this