跳到主要导航 跳到搜索 跳到主要内容

Graph over-parameterization: Why the graph helps the training of deep graph convolutional network

  • Yucong Lin
  • , Silu Li
  • , Jiaxing Xu
  • , Jiawei Xu
  • , Dong Huang
  • , Wendi Zheng
  • , Yuan Cao*
  • , Junwei Lu
  • *此作品的通讯作者
  • Tsinghua University
  • The University of Hong Kong
  • Harvard University

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Recent studies show that gradient descent can train a deep neural network (DNN) to achieve small training and test errors when the DNN is sufficiently wide. This result applies to various over-parameterized neural network models including fully-connected neural networks and convolutional neural networks. However, existing theory does not apply to graph convolutional networks (GCNs), as GCNs is built according to the topological structures of the data. It has been empirically observed that GCNs can outperform vanilla neural networks when the underlying graph captures geometric information of the data. However, there is few theoretical justification of such observation. In this paper, we establish theoretical guarantees of the high-probability convergence of gradient descent for training over-parameterized GCNs. Specifically, we introduce a novel measurement of the relation between the graph and the data, called the “graph disparity coefficient”, and show that the convergence of GCN is faster when the graph disparity coefficient is smaller. Our analysis provides novel insights into how the graph convolution operation in a GCN helps training, and provides useful guidance for GCN training in practice.

源语言英语
页(从-至)77-85
页数9
期刊Neurocomputing
卷534
DOI
出版状态已出版 - 14 5月 2023

学术指纹

探究 'Graph over-parameterization: Why the graph helps the training of deep graph convolutional network' 的科研主题。它们共同构成独一无二的学术指纹。

引用此