On the Performance of STAR-RIS Assisted MIMO-NOMA with GSVD Precoding: An Operator-Valued Free Probability Approach

To meet the capacity requirements of future communications systems, multiple-input multiple-output (MIMO) and non-orthogonal multiple-access (NOMA) are promising techniques to improve the link and system capacities. In addition, the simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) can be well integrated with the communication systems to effectively extend the wireless coverage in a cost-efficient manner. Therefore, in this paper, we investigate the performance of the downlink dual-user MIMO-NOMA systems assisted by STAR-RIS under the generalized singular value decomposition (GSVD) precoding scheme. Specifically, we first apply the operator-valued free probability theory to obtain the Cauchy transform of the generalized singular values (GSVs) of the MIMO-NOMA channel matrices under Rician fading with the Weichselberger's correlation structure, in which a linearization trick for the matrix-valued rational functions is applied to simplify the derivations. Then, based on the Cauchy transform of GSVs, we obtain the closed-form expression for the asymptotic information rates of the users under the considered system. Furthermore, we propose a projected gradient ascent method (PGAM) to enhance the sum rate of the system by designing the phase shifts of the STAR-RIS based on the derived closed-form expressions. The numerical results show the accuracy of the asymptotic expression compared to the Monte Carlo simulations and the effectiveness of the proposed PGAM algorithm.