Robust Tomlinson-Harashima precoding for non-regenerative multi-antenna relaying systems

In this paper, we consider the robust transceiver design with Tomlinson-Harashima precoding (THP) for multi-hop amplify-and-forward (AF) multiple-input multiple-output (MIMO) relaying systems. THP is adopted at the source to mitigate the spatial inter-symbol interference and then a joint Bayesian robust design of THP at source, linear forwarding matrices at relays and linear equalizer at destination is proposed. Based on the elegant characteristics of multiplicative convexity and matrix-monotone functions, the optimal structure of the nonlinear transceiver is first derived. Based on the derived structure, the optimization problem is greatly simplified and can be efficiently solved. Finally, the performance advantage of the proposed robust design is assessed by simulation results.