Low-Cost Video Super-Resolution Assisted by Event Signals

Video super-resolution is a widely studied topic and has been achieving improving its definition in the past decades, especially with the development of deep learning techniques. However, most high-quality super-resolution algorithms are computing-intensive and cannot be readily adopted in computational-resource-limited platforms, such as unmanned aerial vehicles (UAVs). Considering that the fidelity at the contours of moving objects is more important than that in the static background, in terms of both visual quality and quantitative metrics, in this paper, we introduce an event camera to discriminate moving contours with a static background and propose a low-cost patch-specific video super-resolution. We formulate a 0–1 knapsack resource allocation problem to achieve high-quality video super-resolution at a low cost. Specifically, we apply a computing-intensive neural network in the surrounding pixels of moving contours while using a pruned version at the static regions for acceleration. According to the simulation results, the proposed resource allocation algorithm shortens the running time without degeneration of super-resolution performance.