On convergence rate of the randomized Kaczmarz method

For consistent system of linear equations with the coefficient matrix being flat, we conduct an exact closed-form formula for the mean squared error of the iterate generated by the randomized Kaczmarz method, which completes the existing closed-form formula derived only for the tall coefficient matrix. Based upon these formulas, we further estimate an upper bound for the convergence rate of the randomized Kaczmarz method. Both theoretical analysis and numerical experiments demonstrate that this bound can significantly improve the existing ones.