Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

Jinshuo Liu; Yusen Chen; Juan Deng; Donghong Ji; Jeff Pan

doi:10.1007/978-3-319-69005-6_19

Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

Jinshuo Liu, Yusen Chen, Juan Deng, Donghong Ji, Jeff Pan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).

Original language	English
Title of host publication	Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data
Subtitle of host publication	16th China National Conference, CCL 2017, and 5th International Symposium, NLP-NABD 2017, Nanjing, China, October 13-15, 2017, Proceedings
Editors	Maosong Sun, Xiaojie Wang, Baobao Chang, Deyi Xiong
Place of Publication	Cham
Publisher	Springer
Pages	224-236
Number of pages	13
ISBN (Electronic)	978-3-319-69005-6
ISBN (Print)	978-3-319-69005-6
DOIs	https://doi.org/10.1007/978-3-319-69005-6_19
Publication status	Published - 7 Oct 2017
Event	The 16th Chian National Conference on Computational Linguistics & The 5th International Symposium on Natural Processing based on Natural Annoted Big Data - Nanjing, China Duration: 13 Oct 2017 → 15 Oct 2017 http://www.cips-cl.org/static/CCL2017/en/home.html

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer, Cham
Volume	10565
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	The 16th Chian National Conference on Computational Linguistics & The 5th International Symposium on Natural Processing based on Natural Annoted Big Data
Abbreviated title	CCL 2017 and NLP-NABD 2017
Country/Territory	China
City	Nanjing
Period	13/10/17 → 15/10/17
Internet address	http://www.cips-cl.org/static/CCL2017/en/home.html

Keywords / Materials (for Non-textual outputs)

Collaborative recovery
Improved CRF
Chinese abbreviation

Access to Document

10.1007/978-3-319-69005-6_19Licence: All Rights Reserved

https://link.springer.com/chapter/10.1007/978-3-319-69005-6_19Licence: All Rights Reserved

Cite this

Liu, J., Chen, Y., Deng, J., Ji, D., & Pan, J. (2017). Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. In M. Sun, X. Wang, B. Chang, & D. Xiong (Eds.), Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data: 16th China National Conference, CCL 2017, and 5th International Symposium, NLP-NABD 2017, Nanjing, China, October 13-15, 2017, Proceedings (pp. 224-236). (Lecture Notes in Computer Science; Vol. 10565). Springer. https://doi.org/10.1007/978-3-319-69005-6_19

Liu, Jinshuo ; Chen, Yusen ; Deng, Juan et al. / Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data: 16th China National Conference, CCL 2017, and 5th International Symposium, NLP-NABD 2017, Nanjing, China, October 13-15, 2017, Proceedings. editor / Maosong Sun ; Xiaojie Wang ; Baobao Chang ; Deyi Xiong. Cham : Springer, 2017. pp. 224-236 (Lecture Notes in Computer Science).

@inproceedings{ecf646313d2a4454933fbd8c3dd4dd1e,

title = "Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation",

abstract = "One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).",

keywords = "Collaborative recovery, Improved CRF, Chinese abbreviation",

author = "Jinshuo Liu and Yusen Chen and Juan Deng and Donghong Ji and Jeff Pan",

year = "2017",

month = oct,

day = "7",

doi = "10.1007/978-3-319-69005-6_19",

language = "English",

isbn = "978-3-319-69005-6",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "224--236",

editor = "Maosong Sun and Xiaojie Wang and Baobao Chang and Deyi Xiong",

booktitle = "Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data",

address = "United Kingdom",

note = "The 16th Chian National Conference on Computational Linguistics & The 5th International Symposium on Natural Processing based on Natural Annoted Big Data, CCL 2017 and NLP-NABD 2017 ; Conference date: 13-10-2017 Through 15-10-2017",

url = "http://www.cips-cl.org/static/CCL2017/en/home.html",

}

Liu, J, Chen, Y, Deng, J, Ji, D & Pan, J 2017, Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. in M Sun, X Wang, B Chang & D Xiong (eds), Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data: 16th China National Conference, CCL 2017, and 5th International Symposium, NLP-NABD 2017, Nanjing, China, October 13-15, 2017, Proceedings. Lecture Notes in Computer Science, vol. 10565, Springer, Cham, pp. 224-236, The 16th Chian National Conference on Computational Linguistics & The 5th International Symposium on Natural Processing based on Natural Annoted Big Data, Nanjing, China, 13/10/17. https://doi.org/10.1007/978-3-319-69005-6_19

Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. / Liu, Jinshuo; Chen, Yusen; Deng, Juan et al.
Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data: 16th China National Conference, CCL 2017, and 5th International Symposium, NLP-NABD 2017, Nanjing, China, October 13-15, 2017, Proceedings. ed. / Maosong Sun; Xiaojie Wang; Baobao Chang; Deyi Xiong. Cham: Springer, 2017. p. 224-236 (Lecture Notes in Computer Science; Vol. 10565).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

AU - Liu, Jinshuo

AU - Chen, Yusen

AU - Deng, Juan

AU - Ji, Donghong

AU - Pan, Jeff

PY - 2017/10/7

Y1 - 2017/10/7

N2 - One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).

AB - One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).

KW - Collaborative recovery

KW - Improved CRF

KW - Chinese abbreviation

U2 - 10.1007/978-3-319-69005-6_19

DO - 10.1007/978-3-319-69005-6_19

M3 - Conference contribution

SN - 978-3-319-69005-6

T3 - Lecture Notes in Computer Science

SP - 224

EP - 236

BT - Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data

A2 - Sun, Maosong

A2 - Wang, Xiaojie

A2 - Chang, Baobao

A2 - Xiong, Deyi

PB - Springer

CY - Cham

T2 - The 16th Chian National Conference on Computational Linguistics & The 5th International Symposium on Natural Processing based on Natural Annoted Big Data

Y2 - 13 October 2017 through 15 October 2017

ER -

Liu J, Chen Y, Deng J, Ji D, Pan J. Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. In Sun M, Wang X, Chang B, Xiong D, editors, Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data: 16th China National Conference, CCL 2017, and 5th International Symposium, NLP-NABD 2017, Nanjing, China, October 13-15, 2017, Proceedings. Cham: Springer. 2017. p. 224-236. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-69005-6_19

Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this