Transformer-Based Multi-Modal Fusion for Complex Wall Parameter Distribution Estimation

Through-wall radar (TWR) plays a pivotal role in non-intrusive detection of buildings because of its excellent penetrability. Many methods have been developed to estimate the parameters of walls and perform wall compensation for accurate through-wall imaging results. However, most existing studies are based on uniform wall models that cannot be generalized to actual building walls with complex structures such as doors, pillars and windows, nor can they quickly generate a parameter distribution map of the entire wall. To solve this problem, this paper proposes an innovative segment-then-estimate framework that first fuses LiDAR point clouds and optical images with simultaneous localization and mapping (SLAM) to reconstruct a 3D exterior wall model, thereby guiding the precise division of radar B-scan data along the slow-time dimension. Then, we introduce a Transformer-based parameter estimation model for the echo data of each segment, leveraging its self-attention mechanism to capture long-range dependencies and deep features to accurately estimate the wall thickness and relative permittivity. Finally, segment-based estimates are stitched using geometric information obtained from multi-modal reconstruction to generate the wall parameter distribution map. Experimental results demonstrate that the wall parameter estimation maps generated by the proposed framework in this study not only achieve highly accurate parameter estimation values but also exhibit distinct door and window boundaries, providing reliable prior information for highly refined compensated imaging.