AoI and Latency-Aware Air-Ground Vehicular Crowdsensing by Sequential Multi-Agent Deep Reinforcement Learning

Low-latency data sensing and transmission is critical for many city-level applications like traffic incident management to mitigate congestion and enhance road safety. Vehicular crowdsensing (VCS) emerges as a powerful paradigm to provide real-time traffic sensing services from points-of-interest (PoIs) by leveraging the collaboration of unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs). In this paper, we first introduce two novel metrics: sensing capability-aware age-of-information (sAoI) and latency-weighted data collection ratio, to measure the data freshness and amount under the condition of non-uniform status packet size, respectively. We propose an auto-regressive sequential multi-agent deep reinforcement learning framework called “A2G-MADRL”, which consists of an interaction-aware heterogeneous vehicular graph convolution network (HVGCN) for feature extractions, and a dynamically ordered masked policy generator (DOMPG) for coordinating UAVs and UGVs. Extensive experiments on two real-world datasets in KAIST and Roma demonstrate that A2G-MADRL significantly reduces the attained sAoI and improves latency-weighted data collection ratio, outperforming seven baselines when varying the number of UAV-UGV pairs, data generation speed in a timeslot, and the number of communication channels.