A self-consistent-field iteration for MAXBET with an application to multi-view feature extraction

As an extension of the traditional principal component analysis, the multi-view canonical correlation analysis (MCCA) aims at reducing m high dimensional random variables si∈ℝni(i=1,2,…,m) by proper projection matrices Xi∈ℝni×ℓ so that the m reduced ones yi=XiTsi∈ℝℓ have the “maximal correlation.” Various measures of the correlation for y_i (i = 1,2,…,m) in MCCA have been proposed. One of the earliest criteria is the sum of all traces of pair-wise correlation matrices between y_i and y_j subject to the orthogonality constraints on X_i, i = 1,2,…,m. The resulting problem is to maximize a homogeneous quadratic function over the product of Stiefel manifolds and is referred to as the MAXBET problem. In this paper, the problem is first reformulated as a coupled nonlinear eigenvalue problem with eigenvector dependency (NEPv) and then solved by a novel self-consistent-field (SCF) iteration. Global and local convergences of the SCF iteration are studied and proven computational techniques in the standard eigenvalue problem are incorporated to yield more practical implementations. Besides the preliminary numerical evaluations on various types of synthetic problems, the efficiency of the SCF iteration is also demonstrated in an application to multi-view feature extraction for unsupervised learning.