An Efficient Graph Autoencoder with Lightweight Desmoothing Decoder and Long-Range Modeling

Graph self-supervised learning provides a powerful guarantee for learning high-quality representations in an unsupervised manner. Despite its early birth, the performance of generative graph self-supervised learning has long lagged behind that of up-and-coming contrastive learning, especially on node classification tasks. In this paper, we investigate potential issues in existing graph autoencoders and attribute their poor performance to three main aspects: complex decoder design, lack of desmoothing process in feature remap, and overemphasis on local topological proximity. To tackle these issues, we propose an effective and efficient graph autoencoder framework for unsupervised representation learning, which contains two key components: lightweight smoothness-aware feature reconstructor and global structural dependency catcher. After performing a desmoothing operation on encoded representations via a learnable high-pass filter, the feature decoder reconstructs the original features through a simple linear projection. The lightweight design liberates the decoder from self-supervised pretext tasks and puts the encoder more accountable for achieving optimization objectives, which promotes effective training of the encoder. Global structural dependency catcher utilizes graph diffusion to build a structural regularization to capture long-range topological dependency on a graph. The empirical studies demonstrate the effectiveness of our approach, which can surpass dominant contrastive learning methods.