Automatic evaluation of information ordering: Kendall's Tau

This article considers the automatic evaluation of information ordering, a task underlying many text-based applications such as concept-to-text generation and multidocument summarization. We propose an evaluation method based on Kendall's r, a metric of rank correlation. The method is inexpensive, robust, and representation independent. We show that Kendall's T correlates reliably with human ratings and reading times.