Learning Domain Specific Sub-layer Latent Variable for Multi-Domain Adaptation Neural Machine Translation

Shuanghong Huang, Chong Feng*, Ge Shi, Zhengjun Li, Xuan Zhao, Xinyan Li, Xiaomei Wang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Domain adaptation proves to be an effective solution for addressing inadequate translation performance within specific domains. However, the straightforward approach of mixing data from multiple domains to obtain the multi-domain neural machine translation (NMT) model can give rise to the parameter interference between domains problem, resulting in a degradation of overall performance. To address this, we introduce a multi-domain adaptive NMT method aimed at learning domain specific sub-layer latent variable and employ the Gumbel-Softmax reparameterization technique to concurrently train both model parameters and domain specific sub-layer latent variable. This approach facilitates learning private domain-specific knowledge while sharing common domain-invariant knowledge, effectively mitigating the parameter interference problem. The experimental results show that our proposed method significantly improved by up to 7.68 and 3.71 BLEU compared with the baseline model in English-German and Chinese-English public multi-domain datasets, respectively.

Original languageEnglish
Article number78
JournalACM Transactions on Asian and Low-Resource Language Information Processing
Volume23
Issue number6
DOIs
Publication statusPublished - 21 Jun 2024

Keywords

  • multi-domain adaptation
  • Neural machine translation
  • parameter interference
  • sub-layer latent variable

Cite this