Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments

Yiran Suo; Jingfeng Xue; Wenjie Guo; Wenbiao Du; Weijie Han; Chang Xu

doi:10.1007/978-981-96-1525-4_20

Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments

Yiran Suo, Jingfeng Xue^*, Wenjie Guo, Wenbiao Du, Weijie Han^*, Chang Xu

^*Corresponding author for this work

School of Cyberspace Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Encryption technology has become ubiquitous in network communication and encrypted malicious traffic detection becomes an important part of malware detection and cyber attack detection. Existing machine learning models and deep learning models are mainly trained based on packet length sequence information and time series information. Recent studies have shown that these models perform poorly in real network environments. In response to this challenge, this paper proposes a novel malicious traffic detection method based on certificate information extracted during the TLS (Transport Layer Security) encrypted handshake protocol. Our approach demonstrates that certificate information exhibits a strong correlation with the maliciousness of traffic, while remaining unaffected by the complexities of the real network environment. The experimental results illustrate that our method has high accuracy and low time overheading.

Original language	English
Title of host publication	Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings
Editors	Tianqing Zhu, Jin Li, Aniello Castiglione
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	341-350
Number of pages	10
ISBN (Print)	9789819615247
DOIs	https://doi.org/10.1007/978-981-96-1525-4_20
Publication status	Published - 2025
Event	24th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2024 - Macau, China Duration: 29 Oct 2024 → 31 Oct 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	15251 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2024
Country/Territory	China
City	Macau
Period	29/10/24 → 31/10/24

Keywords

Certificate
Encrypted Malicious Traffic Detection
TLS

Access to Document

10.1007/978-981-96-1525-4_20

Cite this

Suo, Y., Xue, J., Guo, W., Du, W., Han, W., & Xu, C. (2025). Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments. In T. Zhu, J. Li, & A. Castiglione (Eds.), Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings (pp. 341-350). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15251 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-96-1525-4_20

Suo, Yiran ; Xue, Jingfeng ; Guo, Wenjie et al. / Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments. Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings. editor / Tianqing Zhu ; Jin Li ; Aniello Castiglione. Springer Science and Business Media Deutschland GmbH, 2025. pp. 341-350 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{76728c1c2dcd4a1184e36d9b15c26b26,

title = "Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments",

abstract = "Encryption technology has become ubiquitous in network communication and encrypted malicious traffic detection becomes an important part of malware detection and cyber attack detection. Existing machine learning models and deep learning models are mainly trained based on packet length sequence information and time series information. Recent studies have shown that these models perform poorly in real network environments. In response to this challenge, this paper proposes a novel malicious traffic detection method based on certificate information extracted during the TLS (Transport Layer Security) encrypted handshake protocol. Our approach demonstrates that certificate information exhibits a strong correlation with the maliciousness of traffic, while remaining unaffected by the complexities of the real network environment. The experimental results illustrate that our method has high accuracy and low time overheading.",

keywords = "Certificate, Encrypted Malicious Traffic Detection, TLS",

author = "Yiran Suo and Jingfeng Xue and Wenjie Guo and Wenbiao Du and Weijie Han and Chang Xu",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.; 24th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2024 ; Conference date: 29-10-2024 Through 31-10-2024",

year = "2025",

doi = "10.1007/978-981-96-1525-4_20",

language = "English",

isbn = "9789819615247",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "341--350",

editor = "Tianqing Zhu and Jin Li and Aniello Castiglione",

booktitle = "Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings",

address = "Germany",

}

Suo, Y, Xue, J, Guo, W, Du, W, Han, W & Xu, C 2025, Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments. in T Zhu, J Li & A Castiglione (eds), Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 15251 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 341-350, 24th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2024, Macau, China, 29/10/24. https://doi.org/10.1007/978-981-96-1525-4_20

Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments. / Suo, Yiran; Xue, Jingfeng; Guo, Wenjie et al.
Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings. ed. / Tianqing Zhu; Jin Li; Aniello Castiglione. Springer Science and Business Media Deutschland GmbH, 2025. p. 341-350 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15251 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments

AU - Suo, Yiran

AU - Xue, Jingfeng

AU - Guo, Wenjie

AU - Du, Wenbiao

AU - Han, Weijie

AU - Xu, Chang

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

PY - 2025

Y1 - 2025

N2 - Encryption technology has become ubiquitous in network communication and encrypted malicious traffic detection becomes an important part of malware detection and cyber attack detection. Existing machine learning models and deep learning models are mainly trained based on packet length sequence information and time series information. Recent studies have shown that these models perform poorly in real network environments. In response to this challenge, this paper proposes a novel malicious traffic detection method based on certificate information extracted during the TLS (Transport Layer Security) encrypted handshake protocol. Our approach demonstrates that certificate information exhibits a strong correlation with the maliciousness of traffic, while remaining unaffected by the complexities of the real network environment. The experimental results illustrate that our method has high accuracy and low time overheading.

AB - Encryption technology has become ubiquitous in network communication and encrypted malicious traffic detection becomes an important part of malware detection and cyber attack detection. Existing machine learning models and deep learning models are mainly trained based on packet length sequence information and time series information. Recent studies have shown that these models perform poorly in real network environments. In response to this challenge, this paper proposes a novel malicious traffic detection method based on certificate information extracted during the TLS (Transport Layer Security) encrypted handshake protocol. Our approach demonstrates that certificate information exhibits a strong correlation with the maliciousness of traffic, while remaining unaffected by the complexities of the real network environment. The experimental results illustrate that our method has high accuracy and low time overheading.

KW - Certificate

KW - Encrypted Malicious Traffic Detection

KW - TLS

UR - http://www.scopus.com/inward/record.url?scp=85218970668&partnerID=8YFLogxK

U2 - 10.1007/978-981-96-1525-4_20

DO - 10.1007/978-981-96-1525-4_20

M3 - Conference contribution

AN - SCOPUS:85218970668

SN - 9789819615247

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 341

EP - 350

BT - Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings

A2 - Zhu, Tianqing

A2 - Li, Jin

A2 - Castiglione, Aniello

PB - Springer Science and Business Media Deutschland GmbH

T2 - 24th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2024

Y2 - 29 October 2024 through 31 October 2024

ER -

Suo Y, Xue J, Guo W, Du W, Han W, Xu C. Certificate-Based Transport Layer Security Encrypted Malicious Traffic Detection in Real-Time Network Environments. In Zhu T, Li J, Castiglione A, editors, Algorithms and Architectures for Parallel Processing - 24th International Conference, ICA3PP 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2025. p. 341-350. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-96-1525-4_20