MDCHD: A novel malware detection method in cloud using hardware trace and deep learning

With the development of cloud computing, more and more enterprises and institutes have deployed important computing tasks and data into virtualization environments. Virtualization security has become very important for cloud computing. When an attacker controls a victim's virtual machine, he (or she) may launch malware for malicious purpose in that virtual machine. To defend against malware attacks in the cloud, many virtualization-based approaches are proposed. However, the existing methods suffer from limitations in terms of transparency and performance cost. To address these issues, we propose MDCHD, a novel malware detection solution for virtualization environments. This method first utilizes the Intel Processor Trace (IPT) mechanism to collect the run-time control flow information of the target program. Then, it converts the control flow information into color images. By doing so, we can utilize a CNN-based deep learning method to identify malware from the images. To improve the performance of our detection mechanism, we leverage Lamport's ring buffer algorithm. In this way, the control flow information collector and security checker can work concurrently. The evaluation shows that our approach can achieve acceptable detection accuracy with a minimal performance cost.