Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation

Fabian Yii; Tom MacGillivray; Miguel O. Bernabeu

doi:10.1007/978-3-031-47425-5_30

Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation

Fabian Yii^*, Tom MacGillivray, Miguel O. Bernabeu

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We investigated the performance of Segment Anything Model (SAM)—the first promptable foundation model for image segmentation—for optic disc (OD) and optic cup (OC) segmentation when fine-tuned on progressively smaller number of fundus images. Three different implementations of SAM with an input prompt were considered: (1) SAM with an OD/OC-centred bounding box (SAM GT); (2) SAM with a noise-added (e.g. displacement, size variation) bounding box (SAM Noise); and (3) SAM with an automatically predicted (using Faster R-CNN) bounding box (SAM Auto). Two popular pre-trained semantic segmentation models, DeepLabV3 with a MobileNetV3-Large backbone and DeepLabV3 with a ResNet-50 backbone were used as baseline models. For OD segmentation, ResNet-50 exhibited comparable if not higher data efficiency (i.e. good performance despite limited training data) than even the most optimal implementation of SAM (SAM GT), although SAM was evidently more robust to small training set sizes, e.g. 25, than MobileNetV3-Large and in eyes with more challenging OD morphologies, e.g. significant peri-papillary atrophy. For OC segmentation, however, SAM GT and SAM Noise consistently demonstrated higher data efficiency, particularly in eyes with relatively small cup-to-disc ratio and ill-defined OC margin.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops
Subtitle of host publication	MTSAIL 2023, LEAF 2023, AI4Treat 2023, MMMI 2023, REMIA 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023, Proceedings
Editors	Jonghye Woo, Alessa Hering, Wilson Silva, Xiang Li, Huazhu Fu, Xiaofeng Liu, Fangxu Xing, Sanjay Purushotham, Tejas S. Mathai, Pritam Mukherjee, Max De Grauw, Regina Beets Tan, Valentina Corbetta, Elmar Kotter, Mauricio Reyes, Christian F. Baumgartner, Quanzheng Li, Richard Leahy, Bin Dong, Hao Chen, Yuankai Huo, Jinglei Lv, Xinxing Xu, Xiaomeng Li, Dwarikanath Mahapatra, Li Cheng, Caroline Petitjean, Benoît Presles
Publisher	Springer
Pages	336-346
Number of pages	11
Volume	14394
ISBN (Electronic)	978-3-031-47425-5
ISBN (Print)	978-3-031-47424-8
DOIs	https://doi.org/10.1007/978-3-031-47425-5_30
Publication status	Published - 3 Feb 2024
Event	26th International Conference on Medical Image Computing and Computer Assisted Intervention - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023 Conference number: 26

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	14394
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Medical Image Computing and Computer Assisted Intervention
Abbreviated title	MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/23 → 12/10/23

Keywords / Materials (for Non-textual outputs)

Segment Anything Model
Optic Disc
Optic Cup

Access to Document

10.1007/978-3-031-47425-5_30

camera_ready_manuscriptAccepted author manuscript, 447 KBLicence: Creative Commons: Attribution (CC-BY)

Cite this

Yii, F., MacGillivray, T., & Bernabeu, M. O. (2024). Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation. In J. Woo, A. Hering, W. Silva, X. Li, H. Fu, X. Liu, F. Xing, S. Purushotham, T. S. Mathai, P. Mukherjee, M. De Grauw, R. B. Tan, V. Corbetta, E. Kotter, M. Reyes, C. F. Baumgartner, Q. Li, R. Leahy, B. Dong, H. Chen, Y. Huo, J. Lv, X. Xu, X. Li, D. Mahapatra, L. Cheng, C. Petitjean, ... B. Presles (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops: MTSAIL 2023, LEAF 2023, AI4Treat 2023, MMMI 2023, REMIA 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023, Proceedings (Vol. 14394, pp. 336-346). (Lecture Notes in Computer Science; Vol. 14394). Springer. https://doi.org/10.1007/978-3-031-47425-5_30

Yii, Fabian ; MacGillivray, Tom ; Bernabeu, Miguel O. / Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops: MTSAIL 2023, LEAF 2023, AI4Treat 2023, MMMI 2023, REMIA 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023, Proceedings. editor / Jonghye Woo ; Alessa Hering ; Wilson Silva ; Xiang Li ; Huazhu Fu ; Xiaofeng Liu ; Fangxu Xing ; Sanjay Purushotham ; Tejas S. Mathai ; Pritam Mukherjee ; Max De Grauw ; Regina Beets Tan ; Valentina Corbetta ; Elmar Kotter ; Mauricio Reyes ; Christian F. Baumgartner ; Quanzheng Li ; Richard Leahy ; Bin Dong ; Hao Chen ; Yuankai Huo ; Jinglei Lv ; Xinxing Xu ; Xiaomeng Li ; Dwarikanath Mahapatra ; Li Cheng ; Caroline Petitjean ; Benoît Presles. Vol. 14394 Springer, 2024. pp. 336-346 (Lecture Notes in Computer Science).

@inproceedings{eb82a02f0d5145b59715ced9a6cef1c5,

title = "Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation",

abstract = "We investigated the performance of Segment Anything Model (SAM)—the first promptable foundation model for image segmentation—for optic disc (OD) and optic cup (OC) segmentation when fine-tuned on progressively smaller number of fundus images. Three different implementations of SAM with an input prompt were considered: (1) SAM with an OD/OC-centred bounding box (SAM GT); (2) SAM with a noise-added (e.g. displacement, size variation) bounding box (SAM Noise); and (3) SAM with an automatically predicted (using Faster R-CNN) bounding box (SAM Auto). Two popular pre-trained semantic segmentation models, DeepLabV3 with a MobileNetV3-Large backbone and DeepLabV3 with a ResNet-50 backbone were used as baseline models. For OD segmentation, ResNet-50 exhibited comparable if not higher data efficiency (i.e. good performance despite limited training data) than even the most optimal implementation of SAM (SAM GT), although SAM was evidently more robust to small training set sizes, e.g. 25, than MobileNetV3-Large and in eyes with more challenging OD morphologies, e.g. significant peri-papillary atrophy. For OC segmentation, however, SAM GT and SAM Noise consistently demonstrated higher data efficiency, particularly in eyes with relatively small cup-to-disc ratio and ill-defined OC margin.",

keywords = "Segment Anything Model, Optic Disc, Optic Cup",

author = "Fabian Yii and Tom MacGillivray and Bernabeu, \{Miguel O.\}",

year = "2024",

month = feb,

day = "3",

doi = "10.1007/978-3-031-47425-5\_30",

language = "English",

isbn = "978-3-031-47424-8",

volume = "14394",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "336--346",

editor = "Woo, \{Jonghye \} and Hering, \{Alessa \} and Silva, \{Wilson \} and Li, \{Xiang \} and Fu, \{Huazhu \} and Xiaofeng Liu and Fangxu Xing and Sanjay Purushotham and Mathai, \{Tejas S.\} and Pritam Mukherjee and \{De Grauw\}, \{Max \} and Tan, \{Regina Beets\} and Corbetta, \{Valentina \} and Elmar Kotter and Mauricio Reyes and Baumgartner, \{Christian F.\} and Quanzheng Li and Richard Leahy and Bin Dong and Hao Chen and Yuankai Huo and Jinglei Lv and Xinxing Xu and Li, \{Xiaomeng \} and Mahapatra, \{Dwarikanath \} and Li Cheng and Petitjean, \{Caroline \} and Presles, \{Beno{\^i}t \}",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops",

address = "United Kingdom",

note = "26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

}

Yii, F , MacGillivray, T & Bernabeu, MO 2024, Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation. in J Woo, A Hering, W Silva, X Li, H Fu, X Liu, F Xing, S Purushotham, TS Mathai, P Mukherjee, M De Grauw, RB Tan, V Corbetta, E Kotter, M Reyes, CF Baumgartner, Q Li, R Leahy, B Dong, H Chen, Y Huo, J Lv, X Xu, X Li, D Mahapatra, L Cheng, C Petitjean & B Presles (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops: MTSAIL 2023, LEAF 2023, AI4Treat 2023, MMMI 2023, REMIA 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023, Proceedings. vol. 14394, Lecture Notes in Computer Science, vol. 14394, Springer, pp. 336-346, 26th International Conference on Medical Image Computing and Computer Assisted Intervention, Vancouver, British Columbia, Canada, 8/10/23. https://doi.org/10.1007/978-3-031-47425-5_30

Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation. / Yii, Fabian ; MacGillivray, Tom ; Bernabeu, Miguel O.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops: MTSAIL 2023, LEAF 2023, AI4Treat 2023, MMMI 2023, REMIA 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023, Proceedings. ed. / Jonghye Woo; Alessa Hering; Wilson Silva; Xiang Li; Huazhu Fu; Xiaofeng Liu; Fangxu Xing; Sanjay Purushotham; Tejas S. Mathai; Pritam Mukherjee; Max De Grauw; Regina Beets Tan; Valentina Corbetta; Elmar Kotter; Mauricio Reyes; Christian F. Baumgartner; Quanzheng Li; Richard Leahy; Bin Dong; Hao Chen; Yuankai Huo; Jinglei Lv; Xinxing Xu; Xiaomeng Li; Dwarikanath Mahapatra; Li Cheng; Caroline Petitjean; Benoît Presles. Vol. 14394 Springer, 2024. p. 336-346 (Lecture Notes in Computer Science; Vol. 14394).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation

AU - Yii, Fabian

AU - MacGillivray, Tom

AU - Bernabeu, Miguel O.

N1 - Conference code: 26

PY - 2024/2/3

Y1 - 2024/2/3

N2 - We investigated the performance of Segment Anything Model (SAM)—the first promptable foundation model for image segmentation—for optic disc (OD) and optic cup (OC) segmentation when fine-tuned on progressively smaller number of fundus images. Three different implementations of SAM with an input prompt were considered: (1) SAM with an OD/OC-centred bounding box (SAM GT); (2) SAM with a noise-added (e.g. displacement, size variation) bounding box (SAM Noise); and (3) SAM with an automatically predicted (using Faster R-CNN) bounding box (SAM Auto). Two popular pre-trained semantic segmentation models, DeepLabV3 with a MobileNetV3-Large backbone and DeepLabV3 with a ResNet-50 backbone were used as baseline models. For OD segmentation, ResNet-50 exhibited comparable if not higher data efficiency (i.e. good performance despite limited training data) than even the most optimal implementation of SAM (SAM GT), although SAM was evidently more robust to small training set sizes, e.g. 25, than MobileNetV3-Large and in eyes with more challenging OD morphologies, e.g. significant peri-papillary atrophy. For OC segmentation, however, SAM GT and SAM Noise consistently demonstrated higher data efficiency, particularly in eyes with relatively small cup-to-disc ratio and ill-defined OC margin.

AB - We investigated the performance of Segment Anything Model (SAM)—the first promptable foundation model for image segmentation—for optic disc (OD) and optic cup (OC) segmentation when fine-tuned on progressively smaller number of fundus images. Three different implementations of SAM with an input prompt were considered: (1) SAM with an OD/OC-centred bounding box (SAM GT); (2) SAM with a noise-added (e.g. displacement, size variation) bounding box (SAM Noise); and (3) SAM with an automatically predicted (using Faster R-CNN) bounding box (SAM Auto). Two popular pre-trained semantic segmentation models, DeepLabV3 with a MobileNetV3-Large backbone and DeepLabV3 with a ResNet-50 backbone were used as baseline models. For OD segmentation, ResNet-50 exhibited comparable if not higher data efficiency (i.e. good performance despite limited training data) than even the most optimal implementation of SAM (SAM GT), although SAM was evidently more robust to small training set sizes, e.g. 25, than MobileNetV3-Large and in eyes with more challenging OD morphologies, e.g. significant peri-papillary atrophy. For OC segmentation, however, SAM GT and SAM Noise consistently demonstrated higher data efficiency, particularly in eyes with relatively small cup-to-disc ratio and ill-defined OC margin.

KW - Segment Anything Model

KW - Optic Disc

KW - Optic Cup

UR - https://doi.org/10.1007/978-3-031-47425-5_30

U2 - 10.1007/978-3-031-47425-5_30

DO - 10.1007/978-3-031-47425-5_30

M3 - Conference contribution

SN - 978-3-031-47424-8

VL - 14394

T3 - Lecture Notes in Computer Science

SP - 336

EP - 346

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops

A2 - Woo, Jonghye

A2 - Hering, Alessa

A2 - Silva, Wilson

A2 - Li, Xiang

A2 - Fu, Huazhu

A2 - Liu, Xiaofeng

A2 - Xing, Fangxu

A2 - Purushotham, Sanjay

A2 - Mathai, Tejas S.

A2 - Mukherjee, Pritam

A2 - De Grauw, Max

A2 - Tan, Regina Beets

A2 - Corbetta, Valentina

A2 - Kotter, Elmar

A2 - Reyes, Mauricio

A2 - Baumgartner, Christian F.

A2 - Li, Quanzheng

A2 - Leahy, Richard

A2 - Dong, Bin

A2 - Chen, Hao

A2 - Huo, Yuankai

A2 - Lv, Jinglei

A2 - Xu, Xinxing

A2 - Li, Xiaomeng

A2 - Mahapatra, Dwarikanath

A2 - Cheng, Li

A2 - Petitjean, Caroline

A2 - Presles, Benoît

PB - Springer

T2 - 26th International Conference on Medical Image Computing and Computer Assisted Intervention

Y2 - 8 October 2023 through 12 October 2023

ER -

Yii F , MacGillivray T , Bernabeu MO. Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation. In Woo J, Hering A, Silva W, Li X, Fu H, Liu X, Xing F, Purushotham S, Mathai TS, Mukherjee P, De Grauw M, Tan RB, Corbetta V, Kotter E, Reyes M, Baumgartner CF, Li Q, Leahy R, Dong B, Chen H, Huo Y, Lv J, Xu X, Li X, Mahapatra D, Cheng L, Petitjean C, Presles B, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops: MTSAIL 2023, LEAF 2023, AI4Treat 2023, MMMI 2023, REMIA 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023, Proceedings. Vol. 14394. Springer. 2024. p. 336-346. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-47425-5_30

Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Image Analysis Core

Cite this

Data Efficiency of Segment Anything Model for Optic Disc and Cup Segmentation

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Equipment

Image Analysis Core

Cite this