Learning to Annotate Part Segmentation with Gradient Matching

Yu Yang, Xiaotian Cheng, Hakan Bilen, Xiangyang Ji

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

The success of state-of-the-art deep neural networks heavily relies on the presence of large-scale labelled datasets, which are extremely expensive and time-consuming to annotate. This paper focuses on tackling semi-supervised part segmentation tasks by generating high-quality images with a pre-trained GAN and labelling the generated images with an automatic annotator. In particular, we formulate the annotator learning as a learning-to-learn problem. Given a pre-trained GAN, the annotator learns to label object parts in a set of randomly generated images such that a part segmentation model trained on these synthetic images with their predicted labels obtains low segmentation error on a small validation set of manually labelled images. We further reduce this nested-loop optimization problem to a simple gradient matching problem and efficiently solve it with an iterative algorithm. We show that our method can learn annotators from a broad range of labelled images including real images, generated images, and even analytically rendered images. Our method is evaluated with semi-supervised part segmentation tasks and significantly outperforms other semi-supervised competitors when the amount of labelled examples is extremely limited.
Original languageEnglish
Title of host publicationProceedings of The Tenth International Conference on Learning Representations
Publication statusPublished - 6 Nov 2022
EventTenth International Conference on Learning Representations 2022 - Virtual Conference
Duration: 25 Apr 202229 Apr 2022
Conference number: 10


ConferenceTenth International Conference on Learning Representations 2022
Abbreviated titleICLR 2022
Internet address

Keywords / Materials (for Non-textual outputs)

  • cs.CV


Dive into the research topics of 'Learning to Annotate Part Segmentation with Gradient Matching'. Together they form a unique fingerprint.

Cite this