ElasticDNN: On-Device Neural Network Remodeling for Adapting Evolving Vision Domains at Edge

Qinglong Zhang, Rui Han*, Chi Harold Liu, Guoren Wang, Lydia Y. Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Executing deep neural networks (DNN) based vision tasks on edge devices encounters challenging scenarios of significant and continually evolving data domains (e.g. background or subpopulation shift). With limited resources, the state-of-the-art domain adaptation (DA) methods either cause high training overheads on large DNN models, or incur significant accuracy losses when adapting small/compressed models in an online fashion. The inefficient resource scheduling among multiple applications further degrades their overall model accuracy. In this paper, we present ElasticDNN, a framework that enables online DNN remodeling for applications encountering evolving domain drifts at edge. Its first key component is the master-surrogate DNN models, which can dynamically generate a small surrogate DNN by retaining and training the large master DNN's most relevant regions pertinent to the new domain. The second novelty of ElasticDNN is the filter-grained resource scheduling, which allocates GPU resources based on online accuracy estimation and DNN remodeling of co-running applications. We fully implement ElasticDNN and demonstrate its effectiveness through extensive experiments. The results show that, compared to existing online DA methods using the same model sizes, ElasticDNN improves accuracy by 23.31% and reduces adaption time by 35.67x. In the more challenging multi-application scenario, ElasticDNN improves accuracy by an average of 25.91%.

Original languageEnglish
Pages (from-to)1616-1630
Number of pages15
JournalIEEE Transactions on Computers
Volume73
Issue number6
DOIs
Publication statusPublished - 1 Jun 2024

Keywords

  • Edge vision
  • deep neural networks
  • domain adaptation

Fingerprint

Dive into the research topics of 'ElasticDNN: On-Device Neural Network Remodeling for Adapting Evolving Vision Domains at Edge'. Together they form a unique fingerprint.

Cite this