Contrastive learning enhanced deep neural network with serial regularization for high-dimensional tabular data

As the scale of data shows rapid growth in various fields, big data's vast amount of information can facilitate scientific discovery or decision-making. Deep neural network prevails in modeling big data such as images and text in computer vision and natural language processing. However, there is currently no widespread deep neural network for high-dimensional tabular data (HTD), as HTD could increase the model's complexity and make estimating the parameters more difficult. Therefore, this paper proposes CLDNSR, a contrastive learning-enhanced deep neural network with serial regularization. This method combines relaxed Bernoulli distribution-based L0 regularization and adaptive L2 regularization for important feature selection and adaptive redundancy control to effectively handle high-dimensional input features. In addition, a tabular contrastive pre-training method is proposed to stabilize the supervised training process through better parameter initialization. Experiments on eleven real-world high-dimensional tabular datasets demonstrate that CLDNSR outperforms the baseline models designed for high-dimensional data.