Human activity recognition with a multibranch network based on CNN and LSTM

Ruixin Yuan, Yanmei Zhang*, Lizhe Wang, Shengyun Li

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

With the widespread use of wearable devices, human activity recognition (HAR) holds immense potential in health monitoring, smart environment. Notably, temporal sensory sequences collected from the wearable devices can provide accurate reflections of the daily activities. Nonetheless, existing CNN-based and LSTM-based methods have predominantly concentrated on feature extraction from univariate sequences, overlooking the implicit frequency information. Therefore, we firstly employed the Short Time Fourier Transform (STFT) in HAR tasks, extracting inherent frequency feature. Concurrently, we introduced a multi-branch network that combines CNN and LSTM. The CNN component captures spatial information of different dimensions. The LSTM, on the other hand, comprises two parts, one focused on temporal relationships within a single channel and the other concerned about channel relationships at a specific time point. In addition, recognizing the limitations in the available datasets, particularly the insufficient coverage of daily activities, we collected our custom dataset, encompassing eight distinct daily activity categories. Finally, we evaluated our proposed model and benchmark models. The results demonstrate that our network exhibits superior generalization across different datasets, archieving accuracy of 91.70%, 95.79%, 87.81% on the PAMAP2, UCI HAR and our own dataset respectively.

Original languageEnglish
Title of host publicationThird International Conference on Computer Technology, Information Engineering, and Electron Materials, CTIEEM 2023
EditorsAtsushi Inoue
PublisherSPIE
ISBN (Electronic)9781510672925
DOIs
Publication statusPublished - 2024
Event3rd International Conference on Computer Technology, Information Engineering, and Electron Materials, CTIEEM 2023 - Zhengzhou, China
Duration: 17 Nov 202319 Nov 2023

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume12987
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

Conference3rd International Conference on Computer Technology, Information Engineering, and Electron Materials, CTIEEM 2023
Country/TerritoryChina
CityZhengzhou
Period17/11/2319/11/23

Keywords

  • cnn
  • deep learning
  • human activity recognition
  • lstm
  • multi-branch

Fingerprint

Dive into the research topics of 'Human activity recognition with a multibranch network based on CNN and LSTM'. Together they form a unique fingerprint.

Cite this