EASTER: Embedding Aggregation-Based Heterogeneous Models Training in Vertical Federated Learning

Vertical Federated Learning (VFL) allows collaborative machine learning without sharing local data. However, existing VFL methods face challenges when dealing with heterogeneous local models among participants, which affects optimization convergence and generalization of participants' local knowledge aggregation. To address this challenge, this paper proposes a novel approach called Embedding Aggregation-based Heterogeneous Models Training in Vertical Federated Learning (EASTER). EASTER focuses on aggregating the local embeddings of each participant's knowledge during forward propagation. We propose an embedding protection method based on lightweight blinding factors, which injects the blinding factors into the local embedding of the passive party. However, the passive party does not own the sample labels, so the local model's gradient cannot be calculated locally. To overcome this limitation, we propose a new method in which the active party assists the passive party in computing its local heterogeneous model gradients. Theoretical analysis and extensive experiments demonstrate that EASTER can simultaneously train multiple heterogeneous models and outperform some recent methods in model performance. For example, compared with the state-of-the-art method, the model accuracy of EASTER was improved by 7.22% under the CIFAR-10 dataset.