Artificial Intelligence-Assisted Design of Imaging Optical Systems (Invited)

Significance The integration of artificial intelligence (AI) with optical systems has achieved notable breakthroughs in optical signal processing, computational imaging, and metasurface design. Traditional optical systems have been developed through iterative manual optimization guided by expert knowledge and numerical methods, requiring substantial time and effort to explore a vast design space. The AI-assisted optical design offers a paradigm shift by enabling data-driven generation of high-quality initial designs and facilitating joint optimization of optical hardware and end-to-end neural networks. The optical design generator treats the neural network parameters as the optimization variables during training, with optical specifications and structural parameters serving as the input and output, respectively. Its goal is to efficiently generate optical systems with good imaging performance with specifications given. The end-to-end optical system joint optimizer takes both the neural network and the optical structural parameters as optimization variables, uses image data as the input and output, and aims to obtain clear images under excellent specifications while allowing the optical system ’s inherent imaging quality to be reduced. In recent years, there have been numerous breakthroughs and advances in this area, yet few studies have systematically reviewed it from the perspective of optical design. It is necessary to summarize the recent developments in AI and computational imaging technologies in light of the requirements of optical design, and to identify the difficulties and challenges in current research. Based on these issues, prospects can be explored for the development of optical design and the potential capabilities of artificial intelligence. Such a review could serve as a valuable reference for researchers intending to enter the field of optical design. Progress This review categorizes AI-assisted optical design into two main directions. Neural network-based optical system generation. We have sorted out the development path of this framework, gradually shifting from the earliest model that required a large amount of data-driven supervised learning to a model that combines unsupervised learning to reduce reliance on the initial dataset. The complexity and optical modulation types of the implemented design are constantly increasing. End-to-end optical-digital joint optimization. The implementation framework and the key position of differentiable optical forward simulation model have been sorted out. Moreover, we have summarized numerous applications of end-to-end joint optical design reported over the past year, such as complex lens design, high specifications design, multi-dimensional light-field capture, design with extended depth of field, and occlusion removal. The paper also examines advanced training strategies such as curriculum learning, hardware-in-the-loop adaptation, and dynamic network architectures, alongside practical considerations like stock lens replacement to reduce cost and fabrication difficulty. Conclusions and Prospects AI-assisted optical design has achieved significant breakthroughs in network architectures, design frameworks, and simulation models, with numerous applied studies continuously emerging. It is destined to become a mainstream driving force in the field of optical design. Future developments will unfold along the following two aspects. For optical design generator, challenges remain in dataset construction, model generalization, training cost, and manufacturability. Future directions include upgrading generation targets, expanding modulation objectives and improving training frameworks. For end-to-end optical-digital joint optimizer, the main challenges lie in computational cost, generalization, interpretability, system versatility, and modulation element performance. Promising research avenues include: integrated optics, hardware-in-the-loop online adaptation, and development of advanced micro/nano-fabricated multidimensional modulation elements.