Performance and complexity analysis of a bidirectional LSTM-based nonlinear equalizer for a terahertz wireless communication system

To address the nonlinear distortion problem caused by terahertz devices, we propose a bidirectional long short-term memory neural network-based nonlinear equalizer (BiLSTM NLE), which is implemented in the offline receiver digital signal processing (DSP) chain. A varying-stride sliding-window mechanism is utilized to reduce the algorithmic computational complexity. The performance of the proposed offline BiLSTM NLE has been experimentally demonstrated in a 220-GHz wireless communication system over a 1-m wireless link. Using quadrature phase-shift keying (QPSK) modulation, the system achieves a transmission baud rate of 16 Gbaud, corresponding to a transmission rate of 32 Gbps. Experimental results show that the proposed BiLSTM NLE effectively mitigates nonlinear distortion and significantly reduces the error vector magnitude (EVM). The equalization performance and computational complexity of the proposed BiLSTM NLE are further compared with those of a traditional Volterra NLE and a baseline convolutional neural network-based nonlinear equalizer (CNN NLE). Compared to the Volterra NLE, the proposed BiLSTM NLE achieves an EVM reduction of over 80%. Furthermore, while achieving comparable EVM performance to the baseline CNN NLE, the proposed BiLSTM NLE reduces both the number of parameters and the real multiplications per equalized symbol (RMpS) by about 96.8%. Our experimental results also verify the cross-condition generalization capability of the proposed BiLSTM NLE under variations in baud rate, arbitrary waveform generator (AWG) output amplitude, and wireless transmission distance.