Transductive Multi-view Embedding for Zero-Shot Recognition and Annotation

Yanwei Fu, Timothy M. Hospedales, Tao Xiang, Zhen-Yong Fu, Shaogang Gong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Most existing zero-shot learning approaches exploit transfer learning via an intermediate-level semantic representation such as visual attributes or semantic word vectors. Such a semantic representation is shared between an annotated auxiliary dataset and a target dataset with no annotation. A projection from a low-level feature space to the semantic space is learned from the auxiliary dataset and is applied without adaptation to the target dataset. In this paper we identify an inherent limitation with this approach. That is, due to having disjoint and potentially unrelated classes, the projection functions learned from the auxiliary dataset/domain are biased when applied directly to the target dataset/domain. We call this problem the projection domain shift problem and propose a novel framework, transductive multi-view embedding, to solve it. It is ‘transductive’ in that unlabelled target data points are explored for projection adaptation, and ‘multi-view’ in that both low-level feature (view) and multiple semantic representations (views) are embedded to rectify the projection shift. We demonstrate through extensive experiments that our framework (1) rectifies the projection shift between the auxiliary and target domains, (2) exploits the complementarity of multiple semantic representations, (3) achieves state-of-the-art recognition results on image and video benchmark datasets, and (4) enables novel cross-view annotation tasks.
Original languageEnglish
Title of host publicationComputer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part II
PublisherSpringer, Cham
Pages584-599
Number of pages16
ISBN (Electronic)978-3-319-10605-2
ISBN (Print)978-3-319-10604-5
DOIs
Publication statusPublished - 2014

Publication series

NameLecture Notes in Computer Science (LNCS)
PublisherSpringer, Cham
Volume8690
ISSN (Print)0302-9743

Fingerprint

Dive into the research topics of 'Transductive Multi-view Embedding for Zero-Shot Recognition and Annotation'. Together they form a unique fingerprint.

Cite this