Weakly Supervised Learning of Objects, Attributes and Their Associations

Zhiyuan Shi, Yongxin Yang, Timothy M. Hospedales, Tao Xiang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

When humans describe images they tend to use combinations of nouns and adjectives, corresponding to objects and their associated attributes respectively. To generate such a description automatically, one needs to model objects, attributes and their associations. Conventional methods require strong annotation of object and attribute locations, making them less scalable. In this paper, we model object-attribute associations from weakly labelled images, such as those widely available on media sharing sites (e.g. Flickr), where only image-level labels (either object or attributes) are given, without their locations and associations. This is achieved by introducing a novel weakly supervised non-parametric Bayesian model. Once learned, given a new image, our model can describe the image, including objects, attributes and their associations, as well as their locations and segmentation. Extensive experiments on benchmark datasets demonstrate that our weakly supervised model performs at par with strongly supervised models on tasks such as image description and retrieval based on object-attribute associations.
Original languageEnglish
Title of host publicationComputer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part II
PublisherSpringer
Pages472-487
Number of pages16
ISBN (Electronic)978-3-319-10605-2
ISBN (Print)978-3-319-10604-5
DOIs
Publication statusPublished - 2014

Publication series

NameLecture Notes in Computer Science (LNCS)
PublisherSpringer, Cham
Volume8690
ISSN (Print)0302-9743

Fingerprint

Dive into the research topics of 'Weakly Supervised Learning of Objects, Attributes and Their Associations'. Together they form a unique fingerprint.

Cite this