Few-shot human motion prediction using deformable spatio-temporal CNN with parameter generation

Human motion prediction is to forecast future pose sequence based on the observed pose sequence. Most recent works rely on large datasets for training. However, the model trained by large datasets cannot be directly applied to the few-shot motion prediction task, which requires the model to generalize well on novel motion dynamics with limited training samples. To deal with this issue, we propose a Motion Prediction Network (MoPredNet) for few shot human motion prediction, which elegantly models long term dependency in motion dynamics and can adapt to predict new forms of motion dynamics. Specifically, the MoPredNet dynamically captures the most informative poses in the data stream as masked poses and adaptively learns spatio-temporal structure from the past poses and the masked poses, and thus improves its encoding capability of motion dynamics. We also propose to cluster training samples into pseudo actions to accumulate prior knowledge, and use the accumulated prior knowledge and few training samples to adapt the MoPredNet to unseen motion dynamics. Experimental results demonstrate that our method achieves better performance over state-of-the-art methods in human motion prediction.