Relational Context Learning for Human-Object Interaction Detection

Interaction action recognition is a popular direction in recent years, as a key technology for understanding human behavior, it plays a crucial role in many fields such as intelligent surveillance and human-computer interaction. Current research often adopts CNN-based or Transformer-based methods, but most of these methods suffer from the problem of lack of context exchange. Aiming at the above problems, the thesis designs and implements a relational context-based interaction action recognition model based on the multivariate relational network based on Transformer method. The main work is as follows: The paper obtains and verifies the runnability of the multivariate relational network (MUREN) source code and explores suitable attention mechanisms for comparison with the original model. First, a channel attention mechanism is used, which, when combined with the MUREN model, achieves a performance improvement of more than 3% on the V-COCO dataset. Tests were conducted on the HICO-DET dataset, which showed that the improved channel attention mechanism was ineffective. Then moving to the use of the global contextual attention mechanism, which matches the relational contextual properties of MUREN, V-COCO is tested only on the HICO-DET dataset since it has already been improved on the channel attention, and the experimental results show an improvement of about 4% in the overall accuracy, and an improvement of about 7% and 6% in the rare samples and the recall rate, respectively.