ReVFed: Representation-Based Privacy-Preserving Vertical Federated Learning with Heterogeneous Models

Vertical Federated Learning (VFL) allows institutions to collaborate on machine learning while keeping their original data private. VFL methods currently concentrate on collaborative training among participants with the same model architectures and use cryptography to protect training parameters. In real application scenarios, participants with different computing resources are more likely to select heterogeneous local models to participate in VFL training automatically. However, current methods face challenges in achieving privacy-preserving VFL with heterogeneous participants. To address the above issues, we propose a novel method called Representation-based Privacy-preserving Vertical Federated Learning with Heterogeneous Models (ReVFed). To reduce the impact of the local model on training, we proposed representation aggregation to incorporate each participant’s local knowledge. Furthermore, we also propose a differential privacy-based protection method to protect local feature representations. Experimental results show that ReVFed effectively ensures privacy-preserving training in VFL with heterogeneous models and delivers excellent performance.