End-to-end active object tracking football game via reinforcement learning

Object detection and tracking in football video is a very challenging task, and it has good practical and commercial value. The traditional method of extracting the target movement trajectory of football matches is often carried out by players carrying recording chips, which is expensive and difficult to popularize in amateur stadiums. There are also some studies that only use the camera to process the targets in the football video, but due to the similar appearance and frequent occlusion of the targets in the football video, these methods can only segment the players and the ball in the image, but cannot. Track it or only for a short period of time. We study active object tracking method for football game, where a tracker takes visual observations (i.e., frame sequences) as input and produces the corresponding camera control signals as output (e.g., turn up, turn left, etc.). Conventional methods tackle tracking and camera control tasks separately, and the resulting system is difficult to tune jointly. These methods also require significant human efforts for image labeling and expensive trial-and-error system tuning in the real world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning. By building a football game simulation scene in the simulator (Unreal Engine), the entire field can be covered by turning the camera in the simulation scene.