Context Embedding Networks

Kun Ho Kim, Oisin Mac Aodha, Pietro Perona

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Low dimensional embeddings that capture the main variations of interest in collections of data are important for many applications. One way to construct these embeddings is to acquire estimates of similarity from the crowd. Similarity is a multi-dimensional concept that varies from individual to individual. However, existing models for learning crowd embeddings typically make simplifying assumptions such as all individuals estimate similarity using the same criteria, the list of criteria is known in advance, or that the crowd workers are not influenced by the data that they see. To overcome these limitations we introduce Context Embedding Networks (CENs). In addition to learning interpretable embeddings from images, CENs also model worker biases for different attributes along with the visual context i.e. the attributes highlighted by a set of images. Experiments on three noisy crowd annotated datasets show that modeling both worker bias and visual context results in more interpretable embeddings compared to existing approaches.
Original languageEnglish
Title of host publication2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages9
ISBN (Electronic)978-1-5386-6420-9
ISBN (Print)978-1-5386-6421-6
Publication statusPublished - 17 Dec 2018
Event2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition - Salt Lake City, United States
Duration: 18 Jun 201822 Jun 2018

Publication series

PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075


Conference2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2018
Country/TerritoryUnited States
CitySalt Lake City
Internet address

Keywords / Materials (for Non-textual outputs)

  • crowdsourcing
  • embedded systems
  • image processing
  • learning (artificial intelligence)
  • low dimensional embeddings
  • crowd workers
  • interpretable embeddings
  • model worker biases
  • noisy crowd
  • visual context
  • context embedding networks
  • CENs
  • data collections
  • crowd similarity
  • multi-dimensional concept
  • crowd embedding learning
  • image set
  • Visualization
  • Context modeling
  • Feature extraction
  • Training
  • Noise measurement
  • Data models
  • Crowdsourcing


Dive into the research topics of 'Context Embedding Networks'. Together they form a unique fingerprint.

Cite this