TY - GEN
T1 - Recognizing Activities from Egocentric Images with Appearance and Motion Features
AU - Chen, Yanhua
AU - Pei, Mingtao
AU - Nie, Zhengang
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - With the development of wearable cameras, recognizing activities from egocentric images has attracted the interest of many researchers. The motion of the camera wearer is an important cue for the activity recognition, and is either explicitly used by optical flow for videos or implicitly used by fusing several images for images. In this paper, based on the observation that the two consecutive images captured by the wearable camera contain the motion information of the camera wearer, we propose to use the camera wearer's rotation and translation computed from the two consecutive images as the motion features. The motion features are combined with appearance features extracted by a CNN as the activity features, and the activity is classified by a random decision forest. We test our method on two egocentric image datasets. The experimental results show that by adding the motion information, the accuracy of activity recognition has been significantly improved.
AB - With the development of wearable cameras, recognizing activities from egocentric images has attracted the interest of many researchers. The motion of the camera wearer is an important cue for the activity recognition, and is either explicitly used by optical flow for videos or implicitly used by fusing several images for images. In this paper, based on the observation that the two consecutive images captured by the wearable camera contain the motion information of the camera wearer, we propose to use the camera wearer's rotation and translation computed from the two consecutive images as the motion features. The motion features are combined with appearance features extracted by a CNN as the activity features, and the activity is classified by a random decision forest. We test our method on two egocentric image datasets. The experimental results show that by adding the motion information, the accuracy of activity recognition has been significantly improved.
KW - Activity Recognition
KW - Camera Motion
KW - Convolutional Neural Networks
KW - Egocentric Image
UR - http://www.scopus.com/inward/record.url?scp=85122833950&partnerID=8YFLogxK
U2 - 10.1109/MLSP52302.2021.9596178
DO - 10.1109/MLSP52302.2021.9596178
M3 - Conference contribution
AN - SCOPUS:85122833950
T3 - IEEE International Workshop on Machine Learning for Signal Processing, MLSP
BT - 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing, MLSP 2021
PB - IEEE Computer Society
T2 - 31st IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2021
Y2 - 25 October 2021 through 28 October 2021
ER -