An efficient CNN accelerator for pattern-compressed sparse neural networks on FPGA

Currently, the sparsity of weights and activations are mainly utilized to improve the energy efficiency and computational performance of CNN accelerators. However, the irregular sparsity of weights and activations can lead to problems such as mapping access conflict and imbalanced workload, which poses great challenges to the efficient computation of sparse networks. To address those problems, this paper proposes a CNN accelerator for pattern-compressed sparse neural networks on FPGA to achieve efficient inference acceleration. It mainly includes three parts: first, we propose a convolutional kernel grouping fusion method to achieve a balanced workload of weights. Second, we propose an activation hash mapping conflict reduction method to drastically reduce the mapping access conflict rate. Finally, we further propose a hierarchical mapping computing architecture to achieve efficient processing of mapping access conflicts and balanced adaptation of weight loads. To verify the effectiveness of the method proposed in this paper, our accelerator is designed and implemented on the Zynq UltraScale+ MPSoC ZCU102 evaluation board and evaluated by running VGG16 and ResNet50 networks. The experimental results show that the method proposed in this paper can reduce the mapping conflicts by more than 97 % and improve the utilization rate of processing element (PE) by 1.67×. Furthermore, the accelerator implemented in this paper can achieve 393.2 GOPs throughput, 1.40–3.25× computational performance, and 1.11–7.34× energy efficiency improvement.