TY - JOUR
T1 - Study Selectively
T2 - 25th Interspeech Conferece 2024
AU - Qiu, Xihang
AU - Zhu, Lixian
AU - Song, Zikai
AU - Chen, Zeyu
AU - Zhang, Haojie
AU - Qian, Kun
AU - Zhang, Ye
AU - Hu, Bin
AU - Yamamoto, Yoshiharu
AU - Schuller, Björn W.
N1 - Publisher Copyright:
© 2024 International Speech Communication Association. All rights reserved.
PY - 2024
Y1 - 2024
N2 - Phonocardiogram classification methods using deep neural networks have been widely applied to the early detection of cardiovascular diseases recently. Despite their excellent recognition rate, the sizeable computational complexity limits their further development. Nowadays, knowledge distillation (KD) is an established paradigm for model compression. While current research on multi-teacher KD has shown potential to impart more comprehensive knowledge to the student than single-teacher KD, this approach is not suitable for all scenarios. This paper proposes a novel KD strategy to realise an adaptive multi-teacher instruction mechanism. We design a teacher selection strategy called voting network to tell the contribution of different teachers on each distillation points, so that the student can choose the useful information and renounce the redundant one. An evaluation demonstrates that our method reaches excellent accuracy (92.8 %) while maintaining a low computational complexity (0.7 M).
AB - Phonocardiogram classification methods using deep neural networks have been widely applied to the early detection of cardiovascular diseases recently. Despite their excellent recognition rate, the sizeable computational complexity limits their further development. Nowadays, knowledge distillation (KD) is an established paradigm for model compression. While current research on multi-teacher KD has shown potential to impart more comprehensive knowledge to the student than single-teacher KD, this approach is not suitable for all scenarios. This paper proposes a novel KD strategy to realise an adaptive multi-teacher instruction mechanism. We design a teacher selection strategy called voting network to tell the contribution of different teachers on each distillation points, so that the student can choose the useful information and renounce the redundant one. An evaluation demonstrates that our method reaches excellent accuracy (92.8 %) while maintaining a low computational complexity (0.7 M).
KW - Adaptive Knowledge Distillation
KW - Computer Audition
KW - Heart Sound
UR - http://www.scopus.com/inward/record.url?scp=85208275180&partnerID=8YFLogxK
U2 - 10.21437/Interspeech.2024-439
DO - 10.21437/Interspeech.2024-439
M3 - Conference article
AN - SCOPUS:85208275180
SN - 2308-457X
SP - 137
EP - 141
JO - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
JF - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Y2 - 1 September 2024 through 5 September 2024
ER -