From human and object interaction to threat detection: An interpretable threat detection method for human violence scenarios

In light of the mounting imperative for public security, the necessity for automated threat detection in high-risk scenarios is becoming increasingly pressing. However, existing methods generally suffer from the problems of uninterpretable inference and biased semantic understanding, which severely limits their reliability in practical deployment. In order to address the aforementioned challenges, this article proposes a threat detection method based on human and object interaction (HOI) tags. This method is based on the fine-grained multimodal dataset, called threat detection by HOI (TD-Hoi), enhancing the model’s semantic modeling ability for key entities and their behavioral interactions by using structured HOI tags to guide language generation. Furthermore, a set of metrics is designed for the evaluation of text response quality, with the objective of systematically measuring the model’s representation accuracy and comprehensibility during threat interpretation. The experimental results have demonstrated that Hoi2Threat attains substantial enhancement in several threat detection tasks, particularly in the core metrics of Correctness of Information, Behavioral Mapping Accuracy, and Threat Detailed Orientation, which are 5.08, 5.04, and 4.76, and 7.10%, 6.80%, and 2.63%, respectively, in comparison with the state-of-the-art method. The aforementioned results provide comprehensive validation of the merits of this approach in the domains of semantic understanding, entity behavior mapping, and interpretability. Ultimately, our work paves the way for more reliable and transparent automated threat detection in real-world security operations.