Max-Fusion U-Net for Multi-Modal Pathology Segmentation with Attention and Dynamic Resampling

Haochuan Jiang, Chengjia Wang, Agisilaos Chartsias, Sotirios Tsaftaris

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Automatic segmentation of multi-sequence (multi-modal) cardiac MR (CMR) images plays a significant role in diagnosis and management for a variety of cardiac diseases. However, the performance of relevant algorithms is significantly affected by the proper fusion of the multi-modal information. Furthermore, particular diseases, such as myocardial infarction, display irregular shapes on images and occupy small regions at random locations. These facts make pathology segmentation of multi-modal CMR images a challenging task. In this paper, we present the Max-Fusion U-Net that achieves improved pathology segmentation performance given aligned multi-modal images of LGE, T2-weighted, and bSSFP modalities. Specifically, modality-specific features are extracted by dedicated encoders. Then they are fused with the pixel-wise maximum operator. Together with the corresponding encoding features, these representations are propagated to decoding layers with U-Net skipconnections. Furthermore, a spatial-attention module is applied in the last decoding layer to encourage the network to focus on those small semantically meaningful pathological regions that trigger relatively high responses by the network neurons. We also use a simple image patch extraction strategy to dynamically resample training examples with varying spacial and batch sizes. With limited GPU memory, this strategy reduces the imbalance of classes and forces the model to focus on regions around the interested pathology. It further improves segmentation accuracy and reduces the mis-classification of pathology. We evaluate our methods using the Myocardial pathology segmentation (MyoPS) combining the multi-sequence CMR dataset which involves three modalities. Extensive experiments demonstrate the effectiveness of the proposed model which outperforms the related baselines. The code is available at
Original languageEnglish
Title of host publicationMyoPS: Myocardial Pathology Segmentation Combining Multi-Sequence CMR Challenge
Subtitle of host publicationMyocardial Pathology Segmentation Combining Multi-Sequence Cardiac Magnetic Resonance Images
ISBN (Electronic)978-3-030-65651-5
ISBN (Print)978-3-030-65650-8
Publication statusE-pub ahead of print - 21 Dec 2020
EventMICCAI 2020 : MyoPS Challenge -
Duration: 4 Sep 20204 Sep 2020

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceMICCAI 2020


  • pathology segmentation
  • Multi-modal
  • max-fusion
  • dynamic resample


Dive into the research topics of 'Max-Fusion U-Net for Multi-Modal Pathology Segmentation with Attention and Dynamic Resampling'. Together they form a unique fingerprint.

Cite this