Keyframe Selection Via Deep Reinforcement Learning for Skeleton-Based Gesture Recognition

Minggang Gan; Jinting Liu; Yuxuan He; Aobo Chen; Qianzhao Ma

doi:10.1109/LRA.2023.3322645

Keyframe Selection Via Deep Reinforcement Learning for Skeleton-Based Gesture Recognition

Minggang Gan^*, Jinting Liu, Yuxuan He, Aobo Chen, Qianzhao Ma

^*Corresponding author for this work

School of Automation

Beijing Institute of Technology

Research output: Contribution to journal › Article › peer-review

1 Citation (Scopus)

Abstract

Skeleton-based gesture recognition has attracted extensive attention and has made great progress. However, mainstream methods generally treat all frames as equally important, which may limit performance, especially when dealing with high inter-class variance in gesture. To tackle this issue, we propose an approach that models a Markov decision process to identify keyframes while discarding irrelevant ones. This article proposes a deep reinforcement learning double-feature double-motion network comprising two main components: a baseline gesture recognition model and a frame selection network. These two components mutually influence each other, resulting in enhanced overall performance. Following the evaluation of the SHREC-17 and F-PHAB datasets, our proposed method demonstrates superior performance.

Original language	English
Pages (from-to)	7807-7814
Number of pages	8
Journal	IEEE Robotics and Automation Letters
Volume	8
Issue number	11
DOIs	https://doi.org/10.1109/LRA.2023.3322645
Publication status	Published - 1 Nov 2023

Keywords

Markov decision process
Skeleton-based gesture recognition
deep reinforcement learning
frame selection network

Access to Document

10.1109/LRA.2023.3322645

Cite this

@article{9a8c34ec8ca24724a5de7e95917c14b9,

title = "Keyframe Selection Via Deep Reinforcement Learning for Skeleton-Based Gesture Recognition",

abstract = "Skeleton-based gesture recognition has attracted extensive attention and has made great progress. However, mainstream methods generally treat all frames as equally important, which may limit performance, especially when dealing with high inter-class variance in gesture. To tackle this issue, we propose an approach that models a Markov decision process to identify keyframes while discarding irrelevant ones. This article proposes a deep reinforcement learning double-feature double-motion network comprising two main components: a baseline gesture recognition model and a frame selection network. These two components mutually influence each other, resulting in enhanced overall performance. Following the evaluation of the SHREC-17 and F-PHAB datasets, our proposed method demonstrates superior performance.",

keywords = "Markov decision process, Skeleton-based gesture recognition, deep reinforcement learning, frame selection network",

author = "Minggang Gan and Jinting Liu and Yuxuan He and Aobo Chen and Qianzhao Ma",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.",

year = "2023",

month = nov,

day = "1",

doi = "10.1109/LRA.2023.3322645",

language = "English",

volume = "8",

pages = "7807--7814",

journal = "IEEE Robotics and Automation Letters",

issn = "2377-3766",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "11",

}

TY - JOUR

T1 - Keyframe Selection Via Deep Reinforcement Learning for Skeleton-Based Gesture Recognition

AU - Gan, Minggang

AU - Liu, Jinting

AU - He, Yuxuan

AU - Chen, Aobo

AU - Ma, Qianzhao

PY - 2023/11/1

Y1 - 2023/11/1

N2 - Skeleton-based gesture recognition has attracted extensive attention and has made great progress. However, mainstream methods generally treat all frames as equally important, which may limit performance, especially when dealing with high inter-class variance in gesture. To tackle this issue, we propose an approach that models a Markov decision process to identify keyframes while discarding irrelevant ones. This article proposes a deep reinforcement learning double-feature double-motion network comprising two main components: a baseline gesture recognition model and a frame selection network. These two components mutually influence each other, resulting in enhanced overall performance. Following the evaluation of the SHREC-17 and F-PHAB datasets, our proposed method demonstrates superior performance.

AB - Skeleton-based gesture recognition has attracted extensive attention and has made great progress. However, mainstream methods generally treat all frames as equally important, which may limit performance, especially when dealing with high inter-class variance in gesture. To tackle this issue, we propose an approach that models a Markov decision process to identify keyframes while discarding irrelevant ones. This article proposes a deep reinforcement learning double-feature double-motion network comprising two main components: a baseline gesture recognition model and a frame selection network. These two components mutually influence each other, resulting in enhanced overall performance. Following the evaluation of the SHREC-17 and F-PHAB datasets, our proposed method demonstrates superior performance.

KW - Markov decision process

KW - Skeleton-based gesture recognition

KW - deep reinforcement learning

KW - frame selection network

UR - http://www.scopus.com/inward/record.url?scp=85174801651&partnerID=8YFLogxK

U2 - 10.1109/LRA.2023.3322645

DO - 10.1109/LRA.2023.3322645

M3 - Article

AN - SCOPUS:85174801651

SN - 2377-3766

VL - 8

SP - 7807

EP - 7814

JO - IEEE Robotics and Automation Letters

JF - IEEE Robotics and Automation Letters

IS - 11

ER -

Keyframe Selection Via Deep Reinforcement Learning for Skeleton-Based Gesture Recognition

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this