Memory Relevant Hyperparameters Optimization for DNN Training at Edge

The full utilization of the low communication latency and privacy-preserving capabilities of edge computing is contingent upon the deployment of an increasing number of deep learning tasks on edge devices for training and inference. However, there are memory-constrained challenges in implementing training models on edge devices. Existing techniques optimise the activation memory for training with lossless accuracy by freezing network parameters, gradient checkpoint, and gradient accumulation. While these methods can also be effective for memory compression, there are fewer memory-integrated optimization techniques for combining multiple techniques to fully utilize the computing power of edge devices, especially when integrating these techniques for edge-side training tasks. In this paper, we propose a Memory Integration Controller (MIC) based on edge-end training resource optimization, which can integrate three complex memory tuning techniques and perform simple tuning through memory-related hyperparameters. The main feature of MIC is that it can provide a lightweight and simple memory tuning means to adapt to the extreme computing environment at the edge under dynamic resource environment. We implement MIC on prevalent deep neural networks (DNNs) and demonstrate its effectiveness against other state-of-the-art techniques. Its integrated memory optimization reduces memory usage by 25x on average, while reducing deployment memory and response time by 28% and 11.8% respectively.