Demystifying Unsupervised Semantic Correspondence Estimation

Mehmet Aygun, Oisin Mac Aodha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We explore semantic correspondence estimation through the lens of unsupervised learning. We thoroughly evaluate several recently proposed unsupervised methods across multiple challenging datasets using a standardized evaluation protocol where we vary factors such as the backbone architecture, the pre-training strategy, and the pre-training and finetuning datasets. To better understand the failure modes of these methods, and in order to provide a clearer path for improvement, we provide a new diagnostic framework along with a new performance metric that is better suited to the semantic matching task. Finally, we introduce a new unsupervised correspondence approach which utilizes the strength of pre-trained features while encouraging better matches during training. This results in significantly better matching performance compared to current state-of-the-art methods.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part X
EditorsShai Avidan, Gabriel Brostow, Moustapha Cissé
PublisherSpringer
Pages125-142
Number of pages31
ISBN (Electronic)978-3-031-20080-9
ISBN (Print)978-3-031-20079-3
DOIs
Publication statusPublished - 23 Oct 2022
EventEuropean Conference on Computer Vision 2022 - Israel, Tel Aviv, Israel
Duration: 23 Oct 202227 Oct 2022
https://eccv2022.ecva.net/

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Cham
Volume13690
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Computer Vision 2022
Abbreviated titleECCV 2022
Country/TerritoryIsrael
CityTel Aviv
Period23/10/2227/10/22
Internet address

Keywords / Materials (for Non-textual outputs)

  • semantic correspondence
  • self-supervised learning

Fingerprint

Dive into the research topics of 'Demystifying Unsupervised Semantic Correspondence Estimation'. Together they form a unique fingerprint.

Cite this