A Method for Anomaly Detection in Surveillance Video Based on Multimodal Large Language Model

With the continuous improvement of urban security systems, intelligent monitoring technology is being widely applied in various fields of social governance. Anomaly detection in surveillance videos plays a crucial role in the development of smart cities. With breakthroughs in crossmodal understanding and reasoning using multimodal large language models, their application in anomaly detection has become a promising new approach. However, directly applying multimodal large language models to surveillance scenarios for anomaly detection using prompt engineering still faces challenges such as significant domain bias in surveillance videos and insufficient output interpretability. To address these challenges, we propose a method called Lo-CoT based on adaptive LoRa fine-tuning and CoT supervised fine-tuning within the surveillance domain, significantly improving the understanding of fine-grained behaviors and the interpretability of anomalies in surveillance scenarios and the accuracy of anomaly detection. We applied our Lo-CoT method to the MSAD dataset and compared it with others' studies, achieving an accuracy improvement of nearly 9 percentage points.