A method of part-of-speech guessing of chinese unknown words based on combined features

Hai Jun Zhang; Shu Min Shi; Chong Feng; He Yan Huang

doi:10.1109/ICMLC.2009.5212477

A method of part-of-speech guessing of chinese unknown words based on combined features

Hai Jun Zhang^*, Shu Min Shi, Chong Feng, He Yan Huang

^*Corresponding author for this work

School of Computer Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (Scopus)

Abstract

Part-Of-Speech (POS) guessing of Unknown Words is an essential phase in the process of Unknown Words Identification. This paper applies combined features (namely, both external and internal features) in POS guessing of Chinese unknown words, under Conditional Random Field model (CRF). For acquiring high-precision of POS guessing, this paper puts forward a method of integrating Chinese radical, as a new internal feature of Chinese characters, into the existing feature set. Experiments show that the application of combined features is effective for POS guessing, and the new feature can significantly improve the performance of POS guessing (precision is up to 94.67%). The results also show that Chinese radical, as an effective internal feature in the field of lexical analysis, has a certain practical value.

Original language	English
Title of host publication	Proceedings of the 2009 International Conference on Machine Learning and Cybernetics
Pages	328-332
Number of pages	5
DOIs	https://doi.org/10.1109/ICMLC.2009.5212477
Publication status	Published - 2009
Event	2009 International Conference on Machine Learning and Cybernetics - Baoding, China Duration: 12 Jul 2009 → 15 Jul 2009

Publication series

Name	Proceedings of the 2009 International Conference on Machine Learning and Cybernetics
Volume	1

Conference

Conference	2009 International Conference on Machine Learning and Cybernetics
Country/Territory	China
City	Baoding
Period	12/07/09 → 15/07/09

Keywords

CRF
Chinese word segmentation
POS guessing
Unknown words

Access to Document

10.1109/ICMLC.2009.5212477

Cite this

Zhang, H. J., Shi, S. M., Feng, C., & Huang, H. Y. (2009). A method of part-of-speech guessing of chinese unknown words based on combined features. In Proceedings of the 2009 International Conference on Machine Learning and Cybernetics (pp. 328-332). Article 5212477 (Proceedings of the 2009 International Conference on Machine Learning and Cybernetics; Vol. 1). https://doi.org/10.1109/ICMLC.2009.5212477

@inproceedings{8a2969c2f09c419aa1bad282c1879e48,

title = "A method of part-of-speech guessing of chinese unknown words based on combined features",

abstract = "Part-Of-Speech (POS) guessing of Unknown Words is an essential phase in the process of Unknown Words Identification. This paper applies combined features (namely, both external and internal features) in POS guessing of Chinese unknown words, under Conditional Random Field model (CRF). For acquiring high-precision of POS guessing, this paper puts forward a method of integrating Chinese radical, as a new internal feature of Chinese characters, into the existing feature set. Experiments show that the application of combined features is effective for POS guessing, and the new feature can significantly improve the performance of POS guessing (precision is up to 94.67%). The results also show that Chinese radical, as an effective internal feature in the field of lexical analysis, has a certain practical value.",

keywords = "CRF, Chinese word segmentation, POS guessing, Unknown words",

author = "Zhang, {Hai Jun} and Shi, {Shu Min} and Chong Feng and Huang, {He Yan}",

year = "2009",

doi = "10.1109/ICMLC.2009.5212477",

language = "English",

isbn = "9781424437030",

series = "Proceedings of the 2009 International Conference on Machine Learning and Cybernetics",

pages = "328--332",

booktitle = "Proceedings of the 2009 International Conference on Machine Learning and Cybernetics",

note = "2009 International Conference on Machine Learning and Cybernetics ; Conference date: 12-07-2009 Through 15-07-2009",

}

Zhang, HJ, Shi, SM , Feng, C & Huang, HY 2009, A method of part-of-speech guessing of chinese unknown words based on combined features. in Proceedings of the 2009 International Conference on Machine Learning and Cybernetics., 5212477, Proceedings of the 2009 International Conference on Machine Learning and Cybernetics, vol. 1, pp. 328-332, 2009 International Conference on Machine Learning and Cybernetics, Baoding, China, 12/07/09. https://doi.org/10.1109/ICMLC.2009.5212477

A method of part-of-speech guessing of chinese unknown words based on combined features. / Zhang, Hai Jun; Shi, Shu Min ; Feng, Chong et al.
Proceedings of the 2009 International Conference on Machine Learning and Cybernetics. 2009. p. 328-332 5212477 (Proceedings of the 2009 International Conference on Machine Learning and Cybernetics; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A method of part-of-speech guessing of chinese unknown words based on combined features

AU - Zhang, Hai Jun

AU - Shi, Shu Min

AU - Feng, Chong

AU - Huang, He Yan

PY - 2009

Y1 - 2009

N2 - Part-Of-Speech (POS) guessing of Unknown Words is an essential phase in the process of Unknown Words Identification. This paper applies combined features (namely, both external and internal features) in POS guessing of Chinese unknown words, under Conditional Random Field model (CRF). For acquiring high-precision of POS guessing, this paper puts forward a method of integrating Chinese radical, as a new internal feature of Chinese characters, into the existing feature set. Experiments show that the application of combined features is effective for POS guessing, and the new feature can significantly improve the performance of POS guessing (precision is up to 94.67%). The results also show that Chinese radical, as an effective internal feature in the field of lexical analysis, has a certain practical value.

AB - Part-Of-Speech (POS) guessing of Unknown Words is an essential phase in the process of Unknown Words Identification. This paper applies combined features (namely, both external and internal features) in POS guessing of Chinese unknown words, under Conditional Random Field model (CRF). For acquiring high-precision of POS guessing, this paper puts forward a method of integrating Chinese radical, as a new internal feature of Chinese characters, into the existing feature set. Experiments show that the application of combined features is effective for POS guessing, and the new feature can significantly improve the performance of POS guessing (precision is up to 94.67%). The results also show that Chinese radical, as an effective internal feature in the field of lexical analysis, has a certain practical value.

KW - CRF

KW - Chinese word segmentation

KW - POS guessing

KW - Unknown words

UR - http://www.scopus.com/inward/record.url?scp=70350728138&partnerID=8YFLogxK

U2 - 10.1109/ICMLC.2009.5212477

DO - 10.1109/ICMLC.2009.5212477

M3 - Conference contribution

AN - SCOPUS:70350728138

SN - 9781424437030

T3 - Proceedings of the 2009 International Conference on Machine Learning and Cybernetics

SP - 328

EP - 332

BT - Proceedings of the 2009 International Conference on Machine Learning and Cybernetics

T2 - 2009 International Conference on Machine Learning and Cybernetics

Y2 - 12 July 2009 through 15 July 2009

ER -

A method of part-of-speech guessing of chinese unknown words based on combined features

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this