TY - GEN
T1 - Part-of-speech tagger based on maximum entropy model
AU - Huang, Heyan
AU - Zhang, Xiaofei
PY - 2009
Y1 - 2009
N2 - The maximum entropy (ME) conditional models don't force to adhere to the independence assumption such as in Hidden Markov generative models, and thus the ME -based Part-of-Speech (POS) tagger can depend on arbitrary, nonindependent features, which are benefit to the POS tagging, without accounting for the distribution of those dependencies. Since ME models are able to flexibly utilize a wide variety of features, the sparse problem of training data is efficiently solved. Experiments show that the POS tagging error rate is reduced by 54.25% in close test and 40.56% in open test over the Hidden-Markov-Model-based baseline, and synchronously an accuracy of 98.01% in close test and 95.56%in open test are obtained.
AB - The maximum entropy (ME) conditional models don't force to adhere to the independence assumption such as in Hidden Markov generative models, and thus the ME -based Part-of-Speech (POS) tagger can depend on arbitrary, nonindependent features, which are benefit to the POS tagging, without accounting for the distribution of those dependencies. Since ME models are able to flexibly utilize a wide variety of features, the sparse problem of training data is efficiently solved. Experiments show that the POS tagging error rate is reduced by 54.25% in close test and 40.56% in open test over the Hidden-Markov-Model-based baseline, and synchronously an accuracy of 98.01% in close test and 95.56%in open test are obtained.
KW - Hidden markov model (HMM)
KW - ME model
KW - Natural language processing (NLP)
KW - POS tagging
UR - http://www.scopus.com/inward/record.url?scp=70449093855&partnerID=8YFLogxK
U2 - 10.1109/ICCSIT.2009.5234787
DO - 10.1109/ICCSIT.2009.5234787
M3 - Conference contribution
AN - SCOPUS:70449093855
SN - 9781424445196
T3 - Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009
SP - 26
EP - 29
BT - Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009
T2 - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009
Y2 - 8 August 2009 through 11 August 2009
ER -