3D semantic mapping based on convolutional neural networks

As an important part of environmental perception, maps guarantee the accuracy of intelligent robots in navigation, localization and path planning. The traditional 3D maps mainly focus on the spatial structure of the objects, which lacks the semantic information. A method is proposed in the paper, this method combines convolutional neural networks (CNNs) and Simultaneous Localization and Mapping (SLAM) to create global dense 3D semantic maps for indoor scenes. The deep neural network that includes convolution and deconvolution is designed to predict semantic category of every pixel. RGB-D camera is used to obtain scene information, accomplish localization and build 3D maps simultaneously. the semantic information is integrated into the 3D scene, we present an octree map method to replace traditional point clouds method, which can reduce the error from pose estimation and single frame labeling. By this method, the accuracy of semantic information is greatly improved.