The Realities of Evaluating Educational Technology in School Settings

Megan Venn-Wycherley; Ahmed Kharrufa; Susan Lechelt; Rebecca Nicholson; Kate Howland; Abrar Almjally; Anthony Trory; Vidya Sarangapani

doi:10.1145/3635146

The Realities of Evaluating Educational Technology in School Settings

Megan Venn-Wycherley, Ahmed Kharrufa, Susan Lechelt, Rebecca Nicholson, Kate Howland, Abrar Almjally, Anthony Trory, Vidya Sarangapani

Research output: Contribution to journal › Article › peer-review

Abstract / Description of output

HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].

Through the analysis of the authors’ contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.

We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

Original language	English
Article number	26
Pages (from-to)	1-33
Journal	ACM Transactions on Computer-Human Interaction
Volume	31
Issue number	2
DOIs	https://doi.org/10.1145/3635146
Publication status	Published - 5 Feb 2024

Keywords / Materials (for Non-textual outputs)

educational technology
classroom evaluationThis work was funded by the EPSRC, including the Centre for Digital Civics award number EP/L016176/1 and EPSRC and BBC iCASE scholarship award number 1623937, and Al Imam Mohammad Ibn Saud Islamic University (IMSIU)..

Access to Document

10.1145/3635146Licence: Creative Commons: Attribution (CC-BY)

The Realities_VENN-WYCHERLEY_DOA10092023_VOR_CC BYFinal published version, 324 KBLicence: Creative Commons: Attribution (CC-BY)

https://dl.acm.org/doi/10.1145/3635146Licence: Creative Commons: Attribution (CC-BY)

Cite this

@article{b52b3473fe1c49e8beb7d78a8868bf5f,

title = "The Realities of Evaluating Educational Technology in School Settings",

abstract = "HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors{\textquoteright} contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.",

keywords = "educational technology, classroom evaluationThis work was funded by the EPSRC, including the Centre for Digital Civics award number EP/L016176/1 and EPSRC and BBC iCASE scholarship award number 1623937, and Al Imam Mohammad Ibn Saud Islamic University (IMSIU)..",

author = "Megan Venn-Wycherley and Ahmed Kharrufa and Susan Lechelt and Rebecca Nicholson and Kate Howland and Abrar Almjally and Anthony Trory and Vidya Sarangapani",

year = "2024",

month = feb,

day = "5",

doi = "10.1145/3635146",

language = "English",

volume = "31",

pages = "1--33",

journal = "ACM Transactions on Computer-Human Interaction",

issn = "1073-0516",

publisher = "Association for Computing Machinery (ACM)",

number = "2",

}

TY - JOUR

T1 - The Realities of Evaluating Educational Technology in School Settings

AU - Venn-Wycherley, Megan

AU - Kharrufa, Ahmed

AU - Lechelt, Susan

AU - Nicholson, Rebecca

AU - Howland, Kate

AU - Almjally, Abrar

AU - Trory, Anthony

AU - Sarangapani, Vidya

PY - 2024/2/5

Y1 - 2024/2/5

N2 - HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors’ contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

AB - HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors’ contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

KW - educational technology

KW - classroom evaluationThis work was funded by the EPSRC, including the Centre for Digital Civics award number EP/L016176/1 and EPSRC and BBC iCASE scholarship award number 1623937, and Al Imam Mohammad Ibn Saud Islamic University (IMSIU)..

U2 - 10.1145/3635146

DO - 10.1145/3635146

M3 - Article

SN - 1073-0516

VL - 31

SP - 1

EP - 33

JO - ACM Transactions on Computer-Human Interaction

JF - ACM Transactions on Computer-Human Interaction

IS - 2

M1 - 26

ER -

The Realities of Evaluating Educational Technology in School Settings

Abstract / Description of output

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this