vMFNet: Compositionality Meets Domain-generalised Segmentation

Xiao Liu, Spyridon Thermos, Pedro Sanchez, Alison O’Neil, Sotirios A. Tsaftaris

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Training medical image segmentation models usually requires a large amount of labeled data. By contrast, humans can quickly learn to accurately recognise anatomy of interest from medical (e.g. MRI and CT) images with some limited guidance. Such recognition ability can easily generalise to new images from different clinical centres. This rapid and generalisable learning ability is mostly due to the compositional structure of image patterns in the human brain, which is less incorporated in medical image segmentation. In this paper, we model the compositional components (i.e. patterns) of human anatomy as learnable von-Mises-Fisher (vMF) kernels, which are robust to images collected from different domains (e.g. clinical centres). The image features can be decomposed to (or composed by) the components with the composing operations, i.e. the vMF likelihoods. The vMF likelihoods tell how likely each anatomical part is at each position of the image. Hence, the segmentation mask can be predicted based on the vMF likelihoods. Moreover, with a reconstruction module, unlabeled data can also be used to learn the vMF kernels and likelihoods by recombining them to reconstruct the input image. Extensive experiments show that the proposed vMFNet achieves improved generalisation performance on two benchmarks, especially when annotations are limited. Code is publicly available at: https://github.com/vios-s/vMFNet.

Original languageUndefined/Unknown
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2022
Subtitle of host publication25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VII
ISBN (Electronic)978-3-031-16449-1
ISBN (Print)978-3-031-16448-4
Publication statusE-pub ahead of print - 17 Sept 2022
Event25th International Conference on Medical Image Computing and Computer Assisted Intervention - Resorts World Sentosa (RWS), Sentosa Island, Singapore
Duration: 18 Sept 202222 Sept 2022

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference25th International Conference on Medical Image Computing and Computer Assisted Intervention
Abbreviated titleMICCAI 2022
CitySentosa Island
Internet address

Keywords / Materials (for Non-textual outputs)

  • Compositionality
  • Domain generalisation
  • Medical image segmentation
  • Semi-supervised learning
  • Test-time training

Cite this