Semantic motion segmentation for urban dynamic scene understanding

A mount of recent researches on scene parsing and semantic labeling, while few focus on obtaining joint semantic motion labeling. In this paper, we propose an approach to infer both the object class and motion status for each pixel of images. First, we extract and match sparse image features to estimate ego-motion between two consecutive stereo images, the result of feature points grouping is used to segment moving object in U-disparity map. Second, a Fully Convolutional Neural Network is employed for semantic segmentation. Moreover, semantic cues are utilized to remove pixels have no potential to be moved in motion mask. Finally, we use a fully connected CRF to integrate motion into semantic segmentation. To validate the effectiveness of the proposed algorithm, we present experimental results with KITTI stereo images that contain moving objects.