What is Healthy? Generative Counterfactual Diffusion for Lesion Localization

Pedro Sanchez; Antanas Kascenas; Xiao Liu; Alison Q. O’Neil; Sotirios A. Tsaftaris

doi:10.1007/978-3-031-18576-2_4

What is Healthy? Generative Counterfactual Diffusion for Lesion Localization

Pedro Sanchez^*, Antanas Kascenas, Xiao Liu, Alison Q. O’Neil, Sotirios A. Tsaftaris

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Reducing the requirement for densely annotated masks in medical image segmentation is important due to cost constraints. In this paper, we consider the problem of inferring pixel-level predictions of brain lesions by only using image-level labels for training. By leveraging recent advances in generative diffusion probabilistic models (DPM), we synthesize counterfactuals of “How would a patient appear if X pathology was not present?”. The difference image between the observed patient state and the healthy counterfactual can be used for inferring the location of pathology. We generate counterfactuals that correspond to the minimal change of the input such that it is transformed to healthy domain. This requires training with healthy and unhealthy data in DPMs. We improve on previous counterfactual DPMs by manipulating the generation process with implicit guidance along with attention conditioning instead of using classifiers (Code is available at https://github.com/vios-s/Diff-SCM ).

Original language	English
Title of host publication	Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings
Editors	Anirban Mukhopadhyay, Ilkay Oksuz, Sandy Engelhardt, Dajiang Zhu, Yixuan Yuan
Publisher	Springer
Pages	34-44
Number of pages	11
ISBN (Print)	9783031185755
DOIs	https://doi.org/10.1007/978-3-031-18576-2_4
Publication status	Published - 8 Oct 2022
Event	2nd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2022, held in Conjunction with the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022 - Singapore, Singapore Duration: 22 Sept 2022 → 22 Sept 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13609 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2nd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2022, held in Conjunction with the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022
Country/Territory	Singapore
City	Singapore
Period	22/09/22 → 22/09/22

Keywords / Materials (for Non-textual outputs)

Counterfactuals
Diffusion probabilistic models
Generative models

Access to Document

10.1007/978-3-031-18576-2_4

Cite this

Sanchez, P., Kascenas, A., Liu, X., O’Neil, A. Q., & Tsaftaris, S. A. (2022). What is Healthy? Generative Counterfactual Diffusion for Lesion Localization. In A. Mukhopadhyay, I. Oksuz, S. Engelhardt, D. Zhu, & Y. Yuan (Eds.), Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings (pp. 34-44). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13609 LNCS). Springer. https://doi.org/10.1007/978-3-031-18576-2_4

Sanchez, Pedro ; Kascenas, Antanas ; Liu, Xiao et al. / What is Healthy? Generative Counterfactual Diffusion for Lesion Localization. Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings. editor / Anirban Mukhopadhyay ; Ilkay Oksuz ; Sandy Engelhardt ; Dajiang Zhu ; Yixuan Yuan. Springer, 2022. pp. 34-44 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b4f23d4d570f4d178f9d6c04a8ca310d,

title = "What is Healthy? Generative Counterfactual Diffusion for Lesion Localization",

abstract = "Reducing the requirement for densely annotated masks in medical image segmentation is important due to cost constraints. In this paper, we consider the problem of inferring pixel-level predictions of brain lesions by only using image-level labels for training. By leveraging recent advances in generative diffusion probabilistic models (DPM), we synthesize counterfactuals of “How would a patient appear if X pathology was not present?”. The difference image between the observed patient state and the healthy counterfactual can be used for inferring the location of pathology. We generate counterfactuals that correspond to the minimal change of the input such that it is transformed to healthy domain. This requires training with healthy and unhealthy data in DPMs. We improve on previous counterfactual DPMs by manipulating the generation process with implicit guidance along with attention conditioning instead of using classifiers (Code is available at https://github.com/vios-s/Diff-SCM ).",

keywords = "Counterfactuals, Diffusion probabilistic models, Generative models",

author = "Pedro Sanchez and Antanas Kascenas and Xiao Liu and O{\textquoteright}Neil, \{Alison Q.\} and Tsaftaris, \{Sotirios A.\}",

note = "Funding Information: Acknowledgements. This work was supported by the University of Edinburgh, the Royal Academy of Engineering and Canon Medical Research Europe via PhD studentships of Pedro Sanchez and Xiao Liu (grant RCSRF1819\textbackslash{}825). This work was partially supported by the Alan Turing Institute under the EPSRC grant EP N510129\textbackslash{}1. Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 2nd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2022, held in Conjunction with the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022 ; Conference date: 22-09-2022 Through 22-09-2022",

year = "2022",

month = oct,

day = "8",

doi = "10.1007/978-3-031-18576-2\_4",

language = "English",

isbn = "9783031185755",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "34--44",

editor = "Anirban Mukhopadhyay and Ilkay Oksuz and Sandy Engelhardt and Dajiang Zhu and Yixuan Yuan",

booktitle = "Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings",

address = "United Kingdom",

}

Sanchez, P, Kascenas, A, Liu, X, O’Neil, AQ & Tsaftaris, SA 2022, What is Healthy? Generative Counterfactual Diffusion for Lesion Localization. in A Mukhopadhyay, I Oksuz, S Engelhardt, D Zhu & Y Yuan (eds), Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13609 LNCS, Springer, pp. 34-44, 2nd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2022, held in Conjunction with the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022, Singapore, Singapore, 22/09/22. https://doi.org/10.1007/978-3-031-18576-2_4

What is Healthy? Generative Counterfactual Diffusion for Lesion Localization. / Sanchez, Pedro; Kascenas, Antanas; Liu, Xiao et al.
Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings. ed. / Anirban Mukhopadhyay; Ilkay Oksuz; Sandy Engelhardt; Dajiang Zhu; Yixuan Yuan. Springer, 2022. p. 34-44 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13609 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - What is Healthy? Generative Counterfactual Diffusion for Lesion Localization

AU - Sanchez, Pedro

AU - Kascenas, Antanas

AU - Liu, Xiao

AU - O’Neil, Alison Q.

AU - Tsaftaris, Sotirios A.

N1 - Funding Information: Acknowledgements. This work was supported by the University of Edinburgh, the Royal Academy of Engineering and Canon Medical Research Europe via PhD studentships of Pedro Sanchez and Xiao Liu (grant RCSRF1819\825). This work was partially supported by the Alan Turing Institute under the EPSRC grant EP N510129\1. Publisher Copyright: © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2022/10/8

Y1 - 2022/10/8

N2 - Reducing the requirement for densely annotated masks in medical image segmentation is important due to cost constraints. In this paper, we consider the problem of inferring pixel-level predictions of brain lesions by only using image-level labels for training. By leveraging recent advances in generative diffusion probabilistic models (DPM), we synthesize counterfactuals of “How would a patient appear if X pathology was not present?”. The difference image between the observed patient state and the healthy counterfactual can be used for inferring the location of pathology. We generate counterfactuals that correspond to the minimal change of the input such that it is transformed to healthy domain. This requires training with healthy and unhealthy data in DPMs. We improve on previous counterfactual DPMs by manipulating the generation process with implicit guidance along with attention conditioning instead of using classifiers (Code is available at https://github.com/vios-s/Diff-SCM ).

AB - Reducing the requirement for densely annotated masks in medical image segmentation is important due to cost constraints. In this paper, we consider the problem of inferring pixel-level predictions of brain lesions by only using image-level labels for training. By leveraging recent advances in generative diffusion probabilistic models (DPM), we synthesize counterfactuals of “How would a patient appear if X pathology was not present?”. The difference image between the observed patient state and the healthy counterfactual can be used for inferring the location of pathology. We generate counterfactuals that correspond to the minimal change of the input such that it is transformed to healthy domain. This requires training with healthy and unhealthy data in DPMs. We improve on previous counterfactual DPMs by manipulating the generation process with implicit guidance along with attention conditioning instead of using classifiers (Code is available at https://github.com/vios-s/Diff-SCM ).

KW - Counterfactuals

KW - Diffusion probabilistic models

KW - Generative models

UR - https://www.scopus.com/pages/publications/85141783219

U2 - 10.1007/978-3-031-18576-2_4

DO - 10.1007/978-3-031-18576-2_4

M3 - Conference contribution

AN - SCOPUS:85141783219

SN - 9783031185755

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 34

EP - 44

BT - Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings

A2 - Mukhopadhyay, Anirban

A2 - Oksuz, Ilkay

A2 - Engelhardt, Sandy

A2 - Zhu, Dajiang

A2 - Yuan, Yixuan

PB - Springer

T2 - 2nd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2022, held in Conjunction with the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022

Y2 - 22 September 2022 through 22 September 2022

ER -

Sanchez P, Kascenas A, Liu X, O’Neil AQ, Tsaftaris SA. What is Healthy? Generative Counterfactual Diffusion for Lesion Localization. In Mukhopadhyay A, Oksuz I, Engelhardt S, Zhu D, Yuan Y, editors, Deep Generative Models - 2nd MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Proceedings. Springer. 2022. p. 34-44. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-18576-2_4

What is Healthy? Generative Counterfactual Diffusion for Lesion Localization

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Canon Medical / RAEng Senior Research Fellow in Healthcare AI

Cite this

What is Healthy? Generative Counterfactual Diffusion for Lesion Localization

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Projects

Canon Medical / RAEng Senior Research Fellow in Healthcare AI

Cite this