网络化系统的学习与控制

A networked system refers to a system composed of multiple subsystems with the capability of interaction and task execution, connected through a network. Due to the properties of high dimension, multiple constraints, non-convexity, and nonlinearity in various application scenarios, the analysis and control of networked systems have received widespread attention from different communities in this century. To address uncertainties in dynamic environments and complex systems, end-to-end methods like reinforcement learning have been introduced to learn control policies of networked systems. However, the high-dimensional nature of networked systems poses significant challenges to the learning efficiency. In fact, many studies have found that the performance of networked systems is often closely related to the network structures. By approaching the problem from a graph perspective, complex optimization and control problems can be transformed into simple combinatorial optimization problems, enabling the scalability of the method to practical large-scale networks. In light of this, this paper systematically reviews learning-based control methods for networked systems from the graph perspective. By examining optimal control problems formulated by linear quadratic regulation and Markov games, respectively, it highlights the critical role of graph structures in learning control policies for networked systems. Additionally, some challenges in this field are outlined and a prospective outlook on future directions is provided.