Heuristic Dual Q-Learning Based Radar Anti-Jamming Decision-Making

With the increasing dynamics and complexity of the modern electronic countermeasure environment, the traditional anti-jamming decision-making methods are difficult to meet the real-time and intelligent demands of cognitive radar and some reinforcement learning algorithms have been designed and used in radar anti-jamming decision-making. However, aiming at the overestimation of Q-value and instability of training results for typical Q-learning, this paper proposes a collaborative optimization based on dual Q-tables and heuristic function for antijamming decision-making in cognitive radar system. It separates the action selection and value evaluation process with the help of two independent Q-tables, thus suppressing the valuation bias caused by the common single Q-table. Meanwhile, the heuristic function is dynamically designed to optimize the exploration process based on the optimal action and reward. Simulation results show that the proposed algorithm improves the decisionmaking accuracy by 16% on average compared with the popular Q-learning and 'State-Action-Reward-State-Action' (Sarsa) methods, and the stability of strategy selection is significantly enhanced, providing a reliable solution for the real-time decisionmaking of cognitive radar system in complex electromagnetic environment.