Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

Keyao Li; Jing Wang; Xin Fu; Xiufeng Sui; Weigong Zhang

doi:10.1007/978-3-030-38961-1_24

Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

Keyao Li, Jing Wang^*, Xin Fu, Xiufeng Sui, Weigong Zhang

^*Corresponding author for this work

School of Management

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Neural networks are increasingly used in recognition, mining and autonomous driving. However, for safety-critical applications, such as autonomous driving, the reliability of NN is an important area that remains largely unexplored. Fortunately, NN itself has fault-tolerance capability, especially, different neurons have different fault-tolerance capability. Thus applying uniform error protection mechanism while ignore this important feature will lead to unnecessary energy and performance overheads. In this paper, we propose a neuron vulnerability factor (NVF) quantifying the neural network vulnerability to soft error, which could provide a good guidance for error-tolerant techniques in NN. Based on NVF, we propose a computation scheduling scheme to reduce the lifetime of neurons with high NVF. The experiment results show that our proposed scheme can improve the accuracy of the neural network by 12% on average, and greatly reduce the fault-tolerant overhead.

Original language	English
Title of host publication	Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings
Editors	Sheng Wen, Albert Zomaya, Laurence T. Yang
Publisher	Springer
Pages	277-285
Number of pages	9
ISBN (Print)	9783030389604
DOIs	https://doi.org/10.1007/978-3-030-38961-1_24
Publication status	Published - 2020
Event	19th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2019 - Melbourne, Australia Duration: 9 Dec 2019 → 11 Dec 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11945 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2019
Country/Territory	Australia
City	Melbourne
Period	9/12/19 → 11/12/19

Keywords

Fault tolerance
Memory protection
Neural network
Neuron Vulnerability Factor
Reliability
Soft error

Access to Document

10.1007/978-3-030-38961-1_24

Cite this

Li, K., Wang, J., Fu, X., Sui, X., & Zhang, W. (2020). Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization. In S. Wen, A. Zomaya, & L. T. Yang (Eds.), Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings (pp. 277-285). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11945 LNCS). Springer. https://doi.org/10.1007/978-3-030-38961-1_24

Li, Keyao ; Wang, Jing ; Fu, Xin et al. / Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization. Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings. editor / Sheng Wen ; Albert Zomaya ; Laurence T. Yang. Springer, 2020. pp. 277-285 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b843fea9c8c44f128b2a9ef58828d5cf,

title = "Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization",

abstract = "Neural networks are increasingly used in recognition, mining and autonomous driving. However, for safety-critical applications, such as autonomous driving, the reliability of NN is an important area that remains largely unexplored. Fortunately, NN itself has fault-tolerance capability, especially, different neurons have different fault-tolerance capability. Thus applying uniform error protection mechanism while ignore this important feature will lead to unnecessary energy and performance overheads. In this paper, we propose a neuron vulnerability factor (NVF) quantifying the neural network vulnerability to soft error, which could provide a good guidance for error-tolerant techniques in NN. Based on NVF, we propose a computation scheduling scheme to reduce the lifetime of neurons with high NVF. The experiment results show that our proposed scheme can improve the accuracy of the neural network by 12% on average, and greatly reduce the fault-tolerant overhead.",

keywords = "Fault tolerance, Memory protection, Neural network, Neuron Vulnerability Factor, Reliability, Soft error",

author = "Keyao Li and Jing Wang and Xin Fu and Xiufeng Sui and Weigong Zhang",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 19th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2019 ; Conference date: 09-12-2019 Through 11-12-2019",

year = "2020",

doi = "10.1007/978-3-030-38961-1_24",

language = "English",

isbn = "9783030389604",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "277--285",

editor = "Sheng Wen and Albert Zomaya and Yang, {Laurence T.}",

booktitle = "Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings",

address = "Germany",

}

Li, K, Wang, J, Fu, X, Sui, X & Zhang, W 2020, Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization. in S Wen, A Zomaya & LT Yang (eds), Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11945 LNCS, Springer, pp. 277-285, 19th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2019, Melbourne, Australia, 9/12/19. https://doi.org/10.1007/978-3-030-38961-1_24

Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization. / Li, Keyao; Wang, Jing; Fu, Xin et al.
Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings. ed. / Sheng Wen; Albert Zomaya; Laurence T. Yang. Springer, 2020. p. 277-285 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11945 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

AU - Li, Keyao

AU - Wang, Jing

AU - Fu, Xin

AU - Sui, Xiufeng

AU - Zhang, Weigong

PY - 2020

Y1 - 2020

N2 - Neural networks are increasingly used in recognition, mining and autonomous driving. However, for safety-critical applications, such as autonomous driving, the reliability of NN is an important area that remains largely unexplored. Fortunately, NN itself has fault-tolerance capability, especially, different neurons have different fault-tolerance capability. Thus applying uniform error protection mechanism while ignore this important feature will lead to unnecessary energy and performance overheads. In this paper, we propose a neuron vulnerability factor (NVF) quantifying the neural network vulnerability to soft error, which could provide a good guidance for error-tolerant techniques in NN. Based on NVF, we propose a computation scheduling scheme to reduce the lifetime of neurons with high NVF. The experiment results show that our proposed scheme can improve the accuracy of the neural network by 12% on average, and greatly reduce the fault-tolerant overhead.

AB - Neural networks are increasingly used in recognition, mining and autonomous driving. However, for safety-critical applications, such as autonomous driving, the reliability of NN is an important area that remains largely unexplored. Fortunately, NN itself has fault-tolerance capability, especially, different neurons have different fault-tolerance capability. Thus applying uniform error protection mechanism while ignore this important feature will lead to unnecessary energy and performance overheads. In this paper, we propose a neuron vulnerability factor (NVF) quantifying the neural network vulnerability to soft error, which could provide a good guidance for error-tolerant techniques in NN. Based on NVF, we propose a computation scheduling scheme to reduce the lifetime of neurons with high NVF. The experiment results show that our proposed scheme can improve the accuracy of the neural network by 12% on average, and greatly reduce the fault-tolerant overhead.

KW - Fault tolerance

KW - Memory protection

KW - Neural network

KW - Neuron Vulnerability Factor

KW - Reliability

KW - Soft error

UR - http://www.scopus.com/inward/record.url?scp=85082132407&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-38961-1_24

DO - 10.1007/978-3-030-38961-1_24

M3 - Conference contribution

AN - SCOPUS:85082132407

SN - 9783030389604

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 277

EP - 285

BT - Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings

A2 - Wen, Sheng

A2 - Zomaya, Albert

A2 - Yang, Laurence T.

PB - Springer

T2 - 19th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2019

Y2 - 9 December 2019 through 11 December 2019

ER -

Li K, Wang J, Fu X, Sui X, Zhang W. Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization. In Wen S, Zomaya A, Yang LT, editors, Algorithms and Architectures for Parallel Processing - 19th International Conference, ICA3PP 2019, Proceedings. Springer. 2020. p. 277-285. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-38961-1_24

Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this