Optimized Mutation of Grey-box Fuzzing: A Deep RL-based Approach

Jiawei Shao, Yan Zhou, Guohua Liu, Dezhi Zheng

科研成果: 书/报告/会议事项章节会议稿件同行评审

1 引用 (Scopus)
Plum Print visual indicator of research metrics
  • Citations
    • Citation Indexes: 1
  • Captures
    • Readers: 4
see details

摘要

As a vulnerability discovery technique, fuzzing has been widely used in the field of software test in the past years. Traditional fuzzing has several drawbacks, including poor efficiency, low code coverage, and a high dependence on expert experience. By introducing the deep reinforcement learning technique, one can train the mutator of the fuzzer to move in a desired direction, such as maximizing code coverage or finding more code paths. This paper proposes a reinforcement learning-based fuzzing method to enhance the code coverage and explore potential code vulnerabilities. First, the concept of the input field is introduced to the seed file, reducing invalid operations by marking whether each byte of the seed file is a valid byte. Then, we optimize mutation by modeling the grey-box fuzzing as a reinforcement learning problem and training mutator's behavior on test cases. By observing the rewards caused by mutating with a specific set of actions performed on an initial program input, the fuzzing agent learns a policy that can next generate new higher-reward inputs. Finally, experimental results show that the proposed deep reinforcement learning-based fuzzing method outperforms the baseline random fuzzing algorithms.

源语言英语
主期刊名Proceedings of 2023 IEEE 12th Data Driven Control and Learning Systems Conference, DDCLS 2023
出版商Institute of Electrical and Electronics Engineers Inc.
1296-1300
页数5
ISBN(电子版)9798350321050
DOI
出版状态已出版 - 2023
活动12th IEEE Data Driven Control and Learning Systems Conference, DDCLS 2023 - Xiangtan, 中国
期限: 12 5月 202314 5月 2023

出版系列

姓名Proceedings of 2023 IEEE 12th Data Driven Control and Learning Systems Conference, DDCLS 2023

会议

会议12th IEEE Data Driven Control and Learning Systems Conference, DDCLS 2023
国家/地区中国
Xiangtan
时期12/05/2314/05/23

指纹

探究 'Optimized Mutation of Grey-box Fuzzing: A Deep RL-based Approach' 的科研主题。它们共同构成独一无二的指纹。

引用此

Shao, J., Zhou, Y., Liu, G., & Zheng, D. (2023). Optimized Mutation of Grey-box Fuzzing: A Deep RL-based Approach. 在 Proceedings of 2023 IEEE 12th Data Driven Control and Learning Systems Conference, DDCLS 2023 (页码 1296-1300). (Proceedings of 2023 IEEE 12th Data Driven Control and Learning Systems Conference, DDCLS 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DDCLS58216.2023.10166955