MRF-MVSNet: A Multiscale Reassigned Feature Network of Multiview Stereo Reconstruction in Remote Sensing

The utilization of Multiview Stereo (MVS) based on deep learning for aerial image depth estimation represents a forefront research area within the remote sensing field. However, issues such as shadows and occlusions persist in remote sensing images impacting the final depth estimation results and current methods lack sufficient discussion on the spatial weight distribution within the cost volume. Consequently, we introduce a multiscale reassigned feature network of multiview stereo reconstruction (MRF-MVSNet) in remote sensing, which aims to effectively filter noise by utilizing high-dimensional features to guide low-dimensional features and incorporates a 3D spatial attention into the cost volume to solve these problems. We introduce Cascade Encoder-decoder Feature Pyramid Networks(CED FPN) for feature extraction within MVSNet, leveraging a where high-dimensional information guides low-dimensional information to further filter noise, thereby yielding superior outcomes in scenarios involving edges, shadows, and other complexities. Furthermore, we propose a type of 3D Spatial Attention Mechanism(3D SAM) to allocate varying weights to the generated cost volume across different spatial locations, ensuring that accurate depth estimates are assigned higher weights. It is worth noting that our MRF-MVSNet achieves a mean absolute error(MAE) of 0.1478 cm, an accuracy of 94.2% for <3-interval(%) and 97.2% for <0.6m(%) on the WHU datasets under the premise of three views as input, which has a significant improvement compared with the existing network.