MalHAPGNN: An Enhanced Call Graph-Based Malware Detection Framework Using Hierarchical Attention Pooling Graph Neural Network

While deep learning techniques have been extensively employed in malware detection, there is a notable challenge in effectively embedding malware features. Current neural network methods primarily capture superficial characteristics, lacking in-depth semantic exploration of functions and failing to preserve structural information at the file level. Motivated by the aforementioned challenges, this paper introduces MalHAPGNN, a novel framework for malware detection that leverages a hierarchical attention pooling graph neural network based on enhanced call graphs. Firstly, to ensure semantic richness, a Bidirectional Encoder Representations from Transformers-based (BERT) attribute-enhanced function embedding method is proposed for the extraction of node attributes in the function call graph. Subsequently, this work designs a hierarchical graph neural network that integrates attention mechanisms and pooling operations, complemented by function node sampling and structural learning strategies. This framework delivers a comprehensive profile of malicious code across semantic, syntactic, and structural dimensions. Extensive experiments conducted on the Kaggle and VirusShare datasets have demonstrated that the proposed framework outperforms other graph neural network (GNN)-based malware detection methods.