Time–frequency spectrogram analysis with adaptive robust fitting for improved lidar-based visibility retrieval

Reliable retrieval of horizontal visibility from pulsed Doppler lidar signals is essential for atmospheric monitoring, but conventional time-domain processing often fails under low signal-to-noise ratios or in complex atmospheric conditions. Here, we present a time–frequency spectrogram–based inversion framework that enhances extinction coefficient boundary determination—a critical step for accurate visibility estimation. The approach applies a modified Douglas–Peucker (DP) algorithm with variance analysis to automatically extract valid linear boundary segments, eliminating manual selection bias. Robust random sample consensus (RANSAC) regression is then employed to estimate the extinction coefficient boundary value, with the residual threshold adaptively optimized via particle swarm optimization (PSO) to maximize inlier counts. Monte Carlo simulations under homogeneous conditions show that the proposed time–frequency method surpasses conventional time-domain techniques in both accuracy and repeatability. Under inhomogeneous atmospheric conditions, simulations incorporating the improved RANSAC algorithm show that time–frequency spectrogram analysis yields nearly a threefold improvement in retrieval accuracy compared to time-domain analysis. These results indicate that coupling time–frequency analysis with adaptive robust fitting offers a powerful and generalizable strategy for precise visibility inversion in complex and variable atmospheric environments.