On convergence rate of the randomized Gauss-Seidel method

The Gauss-Seidel and Kaczmarz methods are two classic iteration methods for solving systems of linear equations, which operate in column and row spaces, respectively. Utilizing the connections between these two methods and imitating the exact analysis of the mean-squared error for the randomized Kaczmarz method, we conduct an exact closed-form formula for the mean-squared residual of the iterate generated by the randomized Gauss-Seidel method. Based on this new formula, we further estimate an upper bound for the convergence rate of the randomized Gauss-Seidel method. Theoretical analysis and numerical experiments show that this bound measurably improves the existing ones. Moreover, these theoretical results are also extended to the more general extrapolated randomized Gauss-Seidel method.