Generalized MSFA Engineering with Structural and Adaptive Nonlocal Demosaicing

The emerging multispectral-filter-array (MSFA) cameras require generalized demosaicing for MSFA engineering. The existing interpolation, compressive sensing and deep learning based methods suffer from either limited reconstruction accuracy or poor generalization. In this work, we report a generalized demosaicing method with structural and adaptive nonlocal optimization, enabling boosted reconstruction accuracy for different MSFAs. The advantages lie in the following three aspects. First, the nonlocal low-rank optimization is applied and extended to the multiple spatial-spectral-temporal dimensions to exploit more crucial details. Second, the block matching accuracy is promoted by employing a novel structural similarity metric instead of the conventional Euclidean distance. Third, the running efficiency is boosted by an adaptive iteration strategy. We built a prototype system to capture raw mosaic images under different MSFAs, and used the technique as an off-the-shelf tool to demonstrate MSFA engineering. The experiments show that the binary tree (BT) based filter array produces higher accuracy than the random and regular ones for different number of channels.