TY - JOUR
T1 - 融合序列语法知识的卷积-自注意力生成式摘要方法
AU - Luo, Senlin
AU - Wang, Ruiyi
AU - Wu, Qian
AU - Pan, Limin
AU - Wu, Zhouting
N1 - Publisher Copyright:
© 2021, Editorial Department of Transaction of Beijing Institute of Technology. All right reserved.
PY - 2021/1
Y1 - 2021/1
N2 - Abstractive summarization is to analyze the core ideas of the document, rephrase or use new words to generate a summary that can summarize the whole document. However, the encoder-decoder model can not fully extract the syntax, that cause the summary not to match the grammar rules. The recurrent neural network is easy to forget the historical information and can not perform parallel computation during training, that cause the main idea of the summary not significant and the coding speed slow. In view of the above problems, a new abstractive summarization method with fusing sequential syntax was proposed for the convolution-self attention model. First, constructing a phrase structure tree for the document and embeding sequential syntax into the encoder, the method could make better use of the syntax when encoding. Then, the convolution-self-attention model was used to replace the recurrent neural network model to encode, learnning the global and local information sufficiently from the document. Experimental results on the CNN/Daily Mail dataset show that, the proposed method is superior to the state-of-the-art methods. At the same time, the generated summaries are more grammatical, the main ideas are more obvious and the encoding speed of the model is faster.
AB - Abstractive summarization is to analyze the core ideas of the document, rephrase or use new words to generate a summary that can summarize the whole document. However, the encoder-decoder model can not fully extract the syntax, that cause the summary not to match the grammar rules. The recurrent neural network is easy to forget the historical information and can not perform parallel computation during training, that cause the main idea of the summary not significant and the coding speed slow. In view of the above problems, a new abstractive summarization method with fusing sequential syntax was proposed for the convolution-self attention model. First, constructing a phrase structure tree for the document and embeding sequential syntax into the encoder, the method could make better use of the syntax when encoding. Then, the convolution-self-attention model was used to replace the recurrent neural network model to encode, learnning the global and local information sufficiently from the document. Experimental results on the CNN/Daily Mail dataset show that, the proposed method is superior to the state-of-the-art methods. At the same time, the generated summaries are more grammatical, the main ideas are more obvious and the encoding speed of the model is faster.
KW - Abstractive summarization
KW - Attention mechanism
KW - Convolution-self attention model
KW - Encoder-decoder model
KW - Grammatical analysis
UR - http://www.scopus.com/inward/record.url?scp=85101357587&partnerID=8YFLogxK
U2 - 10.15918/j.tbit1001-0645.2019.188
DO - 10.15918/j.tbit1001-0645.2019.188
M3 - 文章
AN - SCOPUS:85101357587
SN - 1001-0645
VL - 41
SP - 93
EP - 101
JO - Beijing Ligong Daxue Xuebao/Transaction of Beijing Institute of Technology
JF - Beijing Ligong Daxue Xuebao/Transaction of Beijing Institute of Technology
IS - 1
ER -