PQ-FRL: A Privacy-Preserving and Quality-Aware Federated Reinforcement Learning for UAV-Assisted Edge Computing

Optimizing service performance in UAV-assisted Mobile Edge Computing (MEC) systems often relies on effective cooperative trajectory and resource allocation. Federated Reinforcement Learning (FRL) provides a distributed paradigm to collaboratively learn these policies without sharing raw observations. However, practical deployments face severe coupled challenges: the model quality degradation caused by Differential Privacy (DP) noise injection, heterogeneous environments (Non-IID data), and complex physical constraints. To mitigate these issues, we propose PQ-FRL, a Privacy-Preserving and Quality-Aware FRL framework. Our approach utilizes an Output Perturbation DP mechanism to provide strict per-round protection for uploaded policy updates, representing a pragmatic engineering trade-off between securing immediate operational privacy and maintaining continuous-control DRL convergence. To address the mixed-action space, we incorporate a Straight-Through Estimator (STE) for differentiable offloading decisions, guided by a conditionally shaped reward to prevent suboptimal local equilibria. Furthermore, a novel Quality-Aware (QA) aggregation mechanism dynamically assigns weights based on explicitly DP-protected local reward signals, helping to disentangle noise from inherently poor performance. Simulation results indicate that PQ-FRL can effectively balance realistic non-linear energy and latency constraints. Under the evaluated heterogeneous scenarios and strict privacy budgets, our method demonstrates robust utility preservation and exhibits graceful degradation as physical airspace congestion increases with larger swarm sizes, offering a stable and practical solution for privacy-sensitive UAV edge computing.