Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal

Emanuele Trucco; Alfredo Ruggeri; Thomas Karnowski; Luca Giancardo; Edward Chaum; Jean Pierre Hubschman; Bashir al-Diri; Carol Y. Cheung; Damon Wong; Michael Abramoff; Gilbert Lim; Dinesh Kumar; Philippe Burlina; Neil M. Bressler; Herbert F. Jelinek; Fabrice Meriaudeau; Gwenole Quellec; Tom MacGillivray; Baljean Dhillon

doi:10.1167/iovs.12-10347

Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal

Emanuele Trucco^*, Alfredo Ruggeri, Thomas Karnowski, Luca Giancardo, Edward Chaum, Jean Pierre Hubschman, Bashir al-Diri, Carol Y. Cheung, Damon Wong, Michael Abramoff, Gilbert Lim, Dinesh Kumar, Philippe Burlina, Neil M. Bressler, Herbert F. Jelinek, Fabrice Meriaudeau, Gwenole Quellec, Tom MacGillivray, Baljean Dhillon

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

Abstract / Description of output

This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

Original language	English
Pages (from-to)	3546-3559
Number of pages	14
Journal	Investigative Ophthalmology & Visual Science (IOVS)
Volume	54
Issue number	5
DOIs	https://doi.org/10.1167/iovs.12-10347
Publication status	Published - May 2013

Keywords / Materials (for Non-textual outputs)

validation
fundus image analysis
reference standards
TO-DISC RATIO
DIABETIC-RETINOPATHY
AUTOMATED DETECTION
GLAUCOMA ANALYSIS
MICROANEURYSM DETECTION
MACULAR DEGENERATION
VESSEL SEGMENTATION
COST-EFFECTIVENESS
SCREENING TOOL
CARDIAC CYCLE

Access to Document

10.1167/iovs.12-10347

Cite this

Trucco, E., Ruggeri, A., Karnowski, T., Giancardo, L., Chaum, E., Hubschman, J. P., al-Diri, B., Cheung, C. Y., Wong, D., Abramoff, M., Lim, G., Kumar, D., Burlina, P., Bressler, N. M., Jelinek, H. F., Meriaudeau, F., Quellec, G., MacGillivray, T., & Dhillon, B. (2013). Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal. Investigative Ophthalmology & Visual Science (IOVS), 54(5), 3546-3559. https://doi.org/10.1167/iovs.12-10347

@article{a182972bdbd841f9b51cacbe19d41201,

title = "Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal",

abstract = "This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.",

keywords = "validation, fundus image analysis, reference standards, TO-DISC RATIO, DIABETIC-RETINOPATHY, AUTOMATED DETECTION, GLAUCOMA ANALYSIS, MICROANEURYSM DETECTION, MACULAR DEGENERATION, VESSEL SEGMENTATION, COST-EFFECTIVENESS, SCREENING TOOL, CARDIAC CYCLE",

author = "Emanuele Trucco and Alfredo Ruggeri and Thomas Karnowski and Luca Giancardo and Edward Chaum and Hubschman, {Jean Pierre} and Bashir al-Diri and Cheung, {Carol Y.} and Damon Wong and Michael Abramoff and Gilbert Lim and Dinesh Kumar and Philippe Burlina and Bressler, {Neil M.} and Jelinek, {Herbert F.} and Fabrice Meriaudeau and Gwenole Quellec and Tom MacGillivray and Baljean Dhillon",

year = "2013",

month = may,

doi = "10.1167/iovs.12-10347",

language = "English",

volume = "54",

pages = "3546--3559",

journal = "Investigative Ophthalmology & Visual Science (IOVS)",

issn = "0146-0404",

publisher = "The Association for Research in Vision and Ophthalmology",

number = "5",

}

Trucco, E, Ruggeri, A, Karnowski, T, Giancardo, L, Chaum, E, Hubschman, JP, al-Diri, B, Cheung, CY, Wong, D, Abramoff, M, Lim, G, Kumar, D, Burlina, P, Bressler, NM, Jelinek, HF, Meriaudeau, F, Quellec, G, MacGillivray, T & Dhillon, B 2013, 'Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal', Investigative Ophthalmology & Visual Science (IOVS), vol. 54, no. 5, pp. 3546-3559. https://doi.org/10.1167/iovs.12-10347

TY - JOUR

T1 - Validating Retinal Fundus Image Analysis Algorithms

T2 - Issues and a Proposal

AU - Trucco, Emanuele

AU - Ruggeri, Alfredo

AU - Karnowski, Thomas

AU - Giancardo, Luca

AU - Chaum, Edward

AU - Hubschman, Jean Pierre

AU - al-Diri, Bashir

AU - Cheung, Carol Y.

AU - Wong, Damon

AU - Abramoff, Michael

AU - Lim, Gilbert

AU - Kumar, Dinesh

AU - Burlina, Philippe

AU - Bressler, Neil M.

AU - Jelinek, Herbert F.

AU - Meriaudeau, Fabrice

AU - Quellec, Gwenole

AU - MacGillivray, Tom

AU - Dhillon, Baljean

PY - 2013/5

Y1 - 2013/5

N2 - This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

AB - This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

KW - validation

KW - fundus image analysis

KW - reference standards

KW - TO-DISC RATIO

KW - DIABETIC-RETINOPATHY

KW - AUTOMATED DETECTION

KW - GLAUCOMA ANALYSIS

KW - MICROANEURYSM DETECTION

KW - MACULAR DEGENERATION

KW - VESSEL SEGMENTATION

KW - COST-EFFECTIVENESS

KW - SCREENING TOOL

KW - CARDIAC CYCLE

U2 - 10.1167/iovs.12-10347

DO - 10.1167/iovs.12-10347

M3 - Article

SN - 0146-0404

VL - 54

SP - 3546

EP - 3559

JO - Investigative Ophthalmology & Visual Science (IOVS)

JF - Investigative Ophthalmology & Visual Science (IOVS)

IS - 5

ER -

Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal

Abstract / Description of output

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this