HandAttNet: Attention 3D Hand Mesh Estimation Network

Jintao Sun; Gangyi Ding

doi:10.1109/VRW58643.2023.00165

HandAttNet: Attention 3D Hand Mesh Estimation Network

Jintao Sun, Gangyi Ding

计算机学院

Beijing Institute of Technology

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

Hand pose estimation and reconstruction become increasingly compelling in the metaverse era. But in reality hands are often heavily occluded, which makes the estimation of occluded 3D hand meshes challenging.Previous work tends to ignore the information of the occluded regions, we believe that the occluded regions hand information can be highly utilized, Therefore, in this study, we propose hand mesh estimation network, HandAttNet.We design the cross-attention mechanism module and the DUO-FIT module to inject hand information into the occluded region.Finally, we use the self-attention regression module for 3D hand mesh estimation.Our HandAttNet achieves SOTA performance.

源语言	英语
主期刊名	Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023
出版商	Institute of Electrical and Electronics Engineers Inc.
页	645-646
页数	2
ISBN（电子版）	9798350348392
DOI	https://doi.org/10.1109/VRW58643.2023.00165
出版状态	已出版 - 2023
活动	2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023 - Shanghai, 中国期限: 25 3月 2023 → 29 3月 2023

出版系列

姓名	Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023

会议

会议	2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023
国家/地区	中国
市	Shanghai
时期	25/03/23 → 29/03/23

访问文件

10.1109/VRW58643.2023.00165

其它文件与链接

链接到 Scopus 的出版物

引用此

Sun, J., & Ding, G. (2023). HandAttNet: Attention 3D Hand Mesh Estimation Network. 在 Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023 (页码 645-646). (Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VRW58643.2023.00165

Sun, Jintao ; Ding, Gangyi. / HandAttNet : Attention 3D Hand Mesh Estimation Network. Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. 页码 645-646 (Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023).

@inproceedings{975086f616d74b18804e0e39a4c2517a,

title = "HandAttNet: Attention 3D Hand Mesh Estimation Network",

abstract = "Hand pose estimation and reconstruction become increasingly compelling in the metaverse era. But in reality hands are often heavily occluded, which makes the estimation of occluded 3D hand meshes challenging.Previous work tends to ignore the information of the occluded regions, we believe that the occluded regions hand information can be highly utilized, Therefore, in this study, we propose hand mesh estimation network, HandAttNet.We design the cross-attention mechanism module and the DUO-FIT module to inject hand information into the occluded region.Finally, we use the self-attention regression module for 3D hand mesh estimation.Our HandAttNet achieves SOTA performance.",

author = "Jintao Sun and Gangyi Ding",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023 ; Conference date: 25-03-2023 Through 29-03-2023",

year = "2023",

doi = "10.1109/VRW58643.2023.00165",

language = "English",

series = "Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "645--646",

booktitle = "Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023",

address = "United States",

}

Sun, J & Ding, G 2023, HandAttNet: Attention 3D Hand Mesh Estimation Network. 在 Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023. Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023, Institute of Electrical and Electronics Engineers Inc., 页码 645-646, 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023, Shanghai, 中国, 25/03/23. https://doi.org/10.1109/VRW58643.2023.00165

HandAttNet: Attention 3D Hand Mesh Estimation Network. / Sun, Jintao; Ding, Gangyi.
Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. 页码 645-646 (Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023).

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

TY - GEN

T1 - HandAttNet

T2 - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023

AU - Sun, Jintao

AU - Ding, Gangyi

PY - 2023

Y1 - 2023

N2 - Hand pose estimation and reconstruction become increasingly compelling in the metaverse era. But in reality hands are often heavily occluded, which makes the estimation of occluded 3D hand meshes challenging.Previous work tends to ignore the information of the occluded regions, we believe that the occluded regions hand information can be highly utilized, Therefore, in this study, we propose hand mesh estimation network, HandAttNet.We design the cross-attention mechanism module and the DUO-FIT module to inject hand information into the occluded region.Finally, we use the self-attention regression module for 3D hand mesh estimation.Our HandAttNet achieves SOTA performance.

AB - Hand pose estimation and reconstruction become increasingly compelling in the metaverse era. But in reality hands are often heavily occluded, which makes the estimation of occluded 3D hand meshes challenging.Previous work tends to ignore the information of the occluded regions, we believe that the occluded regions hand information can be highly utilized, Therefore, in this study, we propose hand mesh estimation network, HandAttNet.We design the cross-attention mechanism module and the DUO-FIT module to inject hand information into the occluded region.Finally, we use the self-attention regression module for 3D hand mesh estimation.Our HandAttNet achieves SOTA performance.

UR - http://www.scopus.com/inward/record.url?scp=85159663751&partnerID=8YFLogxK

U2 - 10.1109/VRW58643.2023.00165

DO - 10.1109/VRW58643.2023.00165

M3 - Conference contribution

AN - SCOPUS:85159663751

T3 - Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023

SP - 645

EP - 646

BT - Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 25 March 2023 through 29 March 2023

ER -

Sun J, Ding G. HandAttNet: Attention 3D Hand Mesh Estimation Network. 在 Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023. Institute of Electrical and Electronics Engineers Inc. 2023. 页码 645-646. (Proceedings - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2023). doi: 10.1109/VRW58643.2023.00165

HandAttNet: Attention 3D Hand Mesh Estimation Network

摘要

出版系列

会议

访问文件

其它文件与链接

指纹

引用此