Stop Throwing Away Discriminators! Re-using Adversaries for Test-Time Training

Gabriele Valvano*, Andrea Leo, Sotirios A. Tsaftaris

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Thanks to their ability to learn data distributions without requiring paired data, Generative Adversarial Networks (GANs) have become an integral part of many computer vision methods, including those developed for medical image segmentation. These methods jointly train a segmentor and an adversarial mask discriminator, which provides a data-driven shape prior. At inference, the discriminator is discarded, and only the segmentor is used to predict label maps on test images. But should we discard the discriminator? Here, we argue that the life cycle of adversarial discriminators should not end after training. On the contrary, training stable GANs produces powerful shape priors that we can use to correct segmentor mistakes at inference. To achieve this, we develop stable mask discriminators that do not overfit or catastrophically forget. At test time, we fine-tune the segmentor on each individual test instance until it satisfies the learned shape prior. Our method is simple to implement and increases model performance. Moreover, it opens new directions for re-using mask discriminators at inference. We release the code used for the experiments at https://vios-s.github.io/adversarial-test-time-training.

Original languageEnglish
Title of host publicationDomain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health
Subtitle of host publicationThird MICCAI Workshop, DART 2021, and First MICCAI Workshop, FAIR 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27 and October 1, 2021, Proceedings
EditorsShadi Albarqouni, M. Jorge Cardoso, Qi Dou, Konstantinos Kamnitsas, Bishesh Khanal, Islem Rekik, Nicola Rieke, Debdoot Sheet, Sotirios Tsaftaris, Daguang Xu, Ziyue Xu
PublisherSpringer Science and Business Media Deutschland GmbH
Pages68-78
Number of pages11
ISBN (Print)9783030877217
DOIs
Publication statusPublished - 21 Sep 2021
Event3rd MICCAI Workshop on Domain Adaptation and Representation Transfer, DART 2021, and the 1st MICCAI Workshop on Affordable Healthcare and AI for Resource Diverse Global Health, FAIR 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 - Virtual, Online
Duration: 27 Sep 20211 Oct 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12968 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd MICCAI Workshop on Domain Adaptation and Representation Transfer, DART 2021, and the 1st MICCAI Workshop on Affordable Healthcare and AI for Resource Diverse Global Health, FAIR 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
CityVirtual, Online
Period27/09/211/10/21

Keywords

  • GAN
  • Segmentation
  • Shape prior
  • Test-time training

Fingerprint

Dive into the research topics of 'Stop Throwing Away Discriminators! Re-using Adversaries for Test-Time Training'. Together they form a unique fingerprint.

Cite this