Generative-Adversarial-Network-Enhanced DRL for ISAC With Double Active RISs

integrated sensing and communication (ISAC) is a promising paradigm to alleviate spectrum congestion and facilitate a variety of emerging Internet of Things (IoT) applications. However, the direct links from the ISAC base station (BS) to the users may be blocked due to the obstacles. In this article, we investigate the double-active reconfigurable intelligent surfaces (RISs) assisted ISAC, where two active RISs are used to establish virtual line-of-sight (LoS) links from the ISAC BS to the users. In addition, the sum of the minimum sensing signal-to-interference-plus-noise ratios (SINRs) among multiple targets during a series of time slots is maximized, subject to Quality of Service (QoS) and transmit power constraints, through the joint optimization of transmit, reflection and receive beamforming. We first transform this nonconvex optimization problem in the dynamic environment into a Markov decision process (MDP), and then propose a twin delayed deep deterministic policy gradient (TD3)-based algorithm to solve it. Moreover, to enhance the generalization and stability, we integrate the generative adversarial network (GAN) into the TD3 algorithm and propose a GAN-TD3-based algorithm to handle the beamforming optimization problem. Compared with the TD3-based algorithm, the proposed GAN-TD3-based algorithm achieves the better performance and higher stability at the cost of higher computational complexity and slower convergence speed. Simulation results are presented to verify the effectiveness of our proposed algorithms and the superiority of the active RIS over the passive counterpart.