跳到主要导航 跳到搜索 跳到主要内容

Can Monolingual Pretrained Models Help Cross-Lingual Classification?

  • Beijing Institute of Technology
  • Microsoft USA

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer. However, due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. In this work, we present two approaches to improve zero-shot cross-lingual classification, by transferring the knowledge from monolingual pretrained models to multilingual ones. Experimental results on two cross-lingual classification benchmarks show that our methods outperform vanilla multilingual fine-tuning.

源语言英语
主期刊名Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, AACL-IJCNLP 2020
编辑Kam-Fai Wong, Kevin Knight, Hua Wu
出版商Association for Computational Linguistics (ACL)
页12-17
页数6
ISBN(电子版)9781952148910
DOI
出版状态已出版 - 2020
活动1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, AACL-IJCNLP 2020 - Virtual, Online, 中国
期限: 4 12月 2020 → 7 12月 2020

丛书

姓名Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, AACL-IJCNLP 2020

会议

会议1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, AACL-IJCNLP 2020
国家/地区中国
市Virtual, Online
时期4/12/20 → 7/12/20

学术指纹

探究 'Can Monolingual Pretrained Models Help Cross-Lingual Classification?' 的科研主题。它们共同构成独一无二的学术指纹。

引用此