A Layer-Reduced Neural Network Based Digital Backpropagation Algorithm for Fiber Nonlinearity Mitigation

A layer-reduced neural network based digital backpropagation algorithm called smoothing learned digital backpropagation (smoothing-LDBP), is proposed in this paper. The smoothing-LDBP smooths the power terms in nonlinear activation functions to limit the bandwidth. The limited bandwidth of the power terms generates fewer in-band distortions, thus reduces the required layer for a given equalization performance. Simulation results show that the required layers of smoothing-LDBP are reduced by approximately 62% at 6.7% HD-FEC compared with learned digital backpropagation. Owing to the layer reduction, the latency and the complexity are reduced by 69% and 51%, respectively. The layer-reduced property of smoothing-LDBP is also validated by a proof-of-concept experiment.