Recall is the Proper Evaluation Metric for Word Segmentation

Yan Shao; Christian Hardmeier; Joakim Nivre

Recall is the Proper Evaluation Metric for Word Segmentation

Yan Shao, Christian Hardmeier, Joakim Nivre

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

We extensively analyse the correlations and drawbacks of conventionally employed evaluation metrics for word segmentation. Unlike in standard information retrieval, precision favours under-splitting systems and therefore can be misleading in word segmentation. Overall, based on both theoretical and experimental analysis, we propose that precision should be excluded from the standard evaluation metrics and that the evaluation score obtained by using only recall is sufficient and better correlated with the performance of word segmentation systems.

Original language	English
Title of host publication	Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Place of Publication	Taipei, Taiwan
Publisher	Asian Federation of Natural Language Processing
Pages	86-90
Number of pages	5
Volume	2
ISBN (Electronic)	978-1-948087-01-8
Publication status	Published - 1 Dec 2017
Event	The 8th International Joint Conference on Natural Language Processing - Taipei, Taiwan, Province of China Duration: 27 Nov 2017 → 1 Dec 2017 http://ijcnlp2017.org/site/page.aspx?pid=901&sid=1133&lang=en

Conference

Conference	The 8th International Joint Conference on Natural Language Processing
Abbreviated title	IJCNLP 2017
Country/Territory	Taiwan, Province of China
City	Taipei
Period	27/11/17 → 1/12/17
Internet address	http://ijcnlp2017.org/site/page.aspx?pid=901&sid=1133&lang=en

Access to Document

Recall is the Proper_SHAO_DOA01092017_VOR_CC-BYFinal published version, 166 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/I17-2015Licence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{bf3a49339ca643d18c8367f420fada50,

title = "Recall is the Proper Evaluation Metric for Word Segmentation",

abstract = "We extensively analyse the correlations and drawbacks of conventionally employed evaluation metrics for word segmentation. Unlike in standard information retrieval, precision favours under-splitting systems and therefore can be misleading in word segmentation. Overall, based on both theoretical and experimental analysis, we propose that precision should be excluded from the standard evaluation metrics and that the evaluation score obtained by using only recall is sufficient and better correlated with the performance of word segmentation systems.",

author = "Yan Shao and Christian Hardmeier and Joakim Nivre",

year = "2017",

month = dec,

day = "1",

language = "English",

volume = "2",

pages = "86--90",

booktitle = "Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)",

publisher = "Asian Federation of Natural Language Processing",

note = "The 8th International Joint Conference on Natural Language Processing , IJCNLP 2017 ; Conference date: 27-11-2017 Through 01-12-2017",

url = "http://ijcnlp2017.org/site/page.aspx?pid=901&sid=1133&lang=en",

}

Shao, Y, Hardmeier, C & Nivre, J 2017, Recall is the Proper Evaluation Metric for Word Segmentation. in Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). vol. 2, Asian Federation of Natural Language Processing, Taipei, Taiwan, pp. 86-90, The 8th International Joint Conference on Natural Language Processing , Taipei, Taiwan, Province of China, 27/11/17. <https://www.aclweb.org/anthology/I17-2015>

Recall is the Proper Evaluation Metric for Word Segmentation. / Shao, Yan; Hardmeier, Christian; Nivre, Joakim.
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Vol. 2 Taipei, Taiwan: Asian Federation of Natural Language Processing, 2017. p. 86-90.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Recall is the Proper Evaluation Metric for Word Segmentation

AU - Shao, Yan

AU - Hardmeier, Christian

AU - Nivre, Joakim

PY - 2017/12/1

Y1 - 2017/12/1

N2 - We extensively analyse the correlations and drawbacks of conventionally employed evaluation metrics for word segmentation. Unlike in standard information retrieval, precision favours under-splitting systems and therefore can be misleading in word segmentation. Overall, based on both theoretical and experimental analysis, we propose that precision should be excluded from the standard evaluation metrics and that the evaluation score obtained by using only recall is sufficient and better correlated with the performance of word segmentation systems.

AB - We extensively analyse the correlations and drawbacks of conventionally employed evaluation metrics for word segmentation. Unlike in standard information retrieval, precision favours under-splitting systems and therefore can be misleading in word segmentation. Overall, based on both theoretical and experimental analysis, we propose that precision should be excluded from the standard evaluation metrics and that the evaluation score obtained by using only recall is sufficient and better correlated with the performance of word segmentation systems.

M3 - Conference contribution

VL - 2

SP - 86

EP - 90

BT - Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)

PB - Asian Federation of Natural Language Processing

CY - Taipei, Taiwan

T2 - The 8th International Joint Conference on Natural Language Processing

Y2 - 27 November 2017 through 1 December 2017

ER -

Recall is the Proper Evaluation Metric for Word Segmentation

Abstract / Description of output

Conference

Access to Document

Fingerprint

Cite this