TY - JOUR
T1 - The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision
AU - Cresswell, Kathrin
AU - Rigby, Michael
AU - Magrabi, Farah
AU - Scott, Philip
AU - Brender, Jytte
AU - Craven, Catherine K
AU - Wong, Zoie Shui-Yee
AU - Kukhareva, Polina
AU - Ammenwerth, Elske
AU - Georgiou, Andrew
AU - Medlock, Stephanie
AU - De Keizer, Nicolette F
AU - Nykänen, Pirkko
AU - Prgomet, Mirela
AU - Williams, Robin
N1 - Copyright © 2023 The Author(s). Published by Elsevier B.V. All rights reserved.
PY - 2023/10/1
Y1 - 2023/10/1
N2 - Despite the renewed interest in Artificial Intelligence-based clinical decision support systems (AI-CDS), there is still a lack of empirical evidence supporting their effectiveness. This underscores the need for rigorous and continuous evaluation and monitoring of processes and outcomes associated with the introduction of health information technology. We illustrate how the emergence of AI-CDS has helped to bring to the fore the critical importance of evaluation principles and action regarding all health information technology applications, as these hitherto have received limited attention. Key aspects include assessment of design, implementation and adoption contexts; ensuring systems support and optimise human performance (which in turn requires understanding clinical and system logics); and ensuring that design of systems prioritises ethics, equity, effectiveness, and outcomes. Going forward, information technology strategy, implementation and assessment need to actively incorporate these dimensions. International policy makers, regulators and strategic decision makers in implementing organisations therefore need to be cognisant of these aspects and incorporate them in decision-making and in prioritising investment. In particular, the emphasis needs to be on stronger and more evidence-based evaluation surrounding system limitations and risks as well as optimisation of outcomes, whilst ensuring learning and contextual review. Otherwise, there is a risk that applications will be sub-optimally embodied in health systems with unintended consequences and without yielding intended benefits.
AB - Despite the renewed interest in Artificial Intelligence-based clinical decision support systems (AI-CDS), there is still a lack of empirical evidence supporting their effectiveness. This underscores the need for rigorous and continuous evaluation and monitoring of processes and outcomes associated with the introduction of health information technology. We illustrate how the emergence of AI-CDS has helped to bring to the fore the critical importance of evaluation principles and action regarding all health information technology applications, as these hitherto have received limited attention. Key aspects include assessment of design, implementation and adoption contexts; ensuring systems support and optimise human performance (which in turn requires understanding clinical and system logics); and ensuring that design of systems prioritises ethics, equity, effectiveness, and outcomes. Going forward, information technology strategy, implementation and assessment need to actively incorporate these dimensions. International policy makers, regulators and strategic decision makers in implementing organisations therefore need to be cognisant of these aspects and incorporate them in decision-making and in prioritising investment. In particular, the emphasis needs to be on stronger and more evidence-based evaluation surrounding system limitations and risks as well as optimisation of outcomes, whilst ensuring learning and contextual review. Otherwise, there is a risk that applications will be sub-optimally embodied in health systems with unintended consequences and without yielding intended benefits.
KW - Artificial Intelligence (AI)
KW - Health information technology
KW - ehealth
KW - Evidence
KW - Evaluation
U2 - 10.1016/j.healthpol.2023.104889
DO - 10.1016/j.healthpol.2023.104889
M3 - Article
C2 - 37579545
SN - 1872-6054
VL - 136
JO - Health policy (Amsterdam, Netherlands)
JF - Health policy (Amsterdam, Netherlands)
M1 - 104889
ER -