Hybrid fusion and interpolation algorithm with near-infrared image

Silicon-based digital cameras can record visible and near-infrared (NIR) information, in which the full color visible image (RGB) must be restored from color filter array (CFA) interpolation. In this paper, we propose a unified framework for CFA interpolation and visible/NIR image combination. To obtain a high quality color image, the traditional color interpolation from raw CFA data is improved at each pixel, which is constrained by the corresponding monochromatic NIR image in gradient difference. The experiments indicate the effectiveness of this hybrid scheme to acquire joint color and NIR information in real-time, and show that this hybrid process can generate a better color image when compared to treating interpolation and fusion separately.