System Comparisons: Is There Life after Null?

Natalie B. Steinhauser, Gwendolyn E. Campbell, Sarah Dehne, Myroslava O. Dzikovska, Johanna D. Moore

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

It is common practice to compare gain scores in order to determine the effectiveness of adding features to a training system. Here we argue that relying on one measure of overall system effectiveness may result in overlooking valuable lessons available from a comparison of different versions of a system. To illustrate our point, we present the results of comparing a Natural Language Processing (NLP) based adaptive feedback system to a system that does not utilize NLP capabilities. We show that, while there were no learning gain differences between the two systems, the correlates to gain were different. In the non-NLP system, only student performance during the training was correlated to learning gain. In the adaptive system, more variables correlated with learning, including measures of system capability and student satisfaction. This level of analysis suggests that the two systems are not equivalent and points us towards modifications that may improve effectiveness.
Original languageEnglish
Title of host publicationArtificial Intelligence in Education
Subtitle of host publication16th International Conference, AIED 2013, Memphis, TN, USA, July 9-13, 2013. Proceedings
EditorsH. Chad Lane, Kalina Yacef, Jack Mostow, Philip Pavlik
PublisherSpringer-Verlag GmbH
Number of pages4
ISBN (Electronic)978-3-642-39112-5
ISBN (Print)978-3-642-39111-8
Publication statusPublished - 2013

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Berlin / Heidelberg
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords / Materials (for Non-textual outputs)

  • Intelligent Tutoring Systems
  • Adaptive feedback
  • Natural Language Processing
  • Effectiveness evaluation


Dive into the research topics of 'System Comparisons: Is There Life after Null?'. Together they form a unique fingerprint.

Cite this