Temporal Difference Enhancement for High-Resolution Video Frame Interpolation

Video frame interpolation techniques provide a smoother visual experience by enhancing the temporal resolution of videos. To generate intermediate frames, numerous techniques estimate various parameters, such as optical flow and occlusion masks, directly on the original resolution images. As a result, processing high-resolution images requires more computing power and inference time. This paper proposes a lightweight network for high-resolution video frame interpolation that performs a complete interpolation workflow on low-resolution images to provide accurate low-resolution optical flow and occlusion masks. To effectively restore the optical flow and mask of the original resolution image, we propose an extremely lightweight temporal difference enhancement module that makes use of the hidden motion information in the temporal difference to aid in the restoration of optical flow and mask. The proposed network has comparable performance and faster inference speed for high-resolution video interpolation compared to the current mainstream network. The ablation experiment demonstrates the importance of the temporal difference module.