Exploring the use of computational linguistics for automated formative feedback in the humanities

This paper presents the results an investigation into the possible role of computational linguistic techniques in providing automated formative feedback on student's written work - including both traditional essays, and collaborative wikis.

We first attempt to identify some of the criteria used by academic staff when marking student work. Using real examples from the School of Divinity at Edinburgh University, we analyse written feedback from the markers to produce an explicit list of criteria. We also use a number of automated tools to analyse certain features of the work and identify those which correlate with the perceived "quality".

We then survey various techniques from computational linguistics to determine how they might potentially be used to identify some of these criteria automatically.

We conclude that there is a real potential to produce an automated tool which would provide practically useful formative feedback. We also note that the process of rigorously defining the criteria was helpful to the academic staff in clarifying their manual marking process.