An In-depth Interactive and Visualized Platform for Evaluating and Analyzing MRC Models

Zhijing Wu, Jingliang Fang, Hua Xu*, Kai Gao

*此作品的通讯作者

科研成果: 书/报告/会议事项章节会议稿件同行评审

摘要

Machine Reading Comprehension (MRC) has made leaps and bounds when focusing on answering questions. However, since the existing accuracy-based evaluation metrics are agnostic to the nuances of neural networks, the true understanding and inferencing abilities of MRC models remain largely unknown. To address the above limitations, InDepth-Eva-MRC, an interactive and visualized platform, is proposed to provide analysis from cognitive fine-grained for MRC models. Concretely, the platform makes post-hoc systems to explain the behavior of MRC models. On the one hand, it analyzes the linguistic bias via performances with different linguistic properties. On the other hand, it performs skill-based analysis methods based on the modified test samples and semi-automatically generated test samples. Furthermore, through its detailed and interactive visualizations, the platform offers in-depth results analysis and model comparison from cognitive fine-grained. A screencast video and additional external material are available on https://github.com/thuiar/InDepth-Eva-MRC.

源语言英语
主期刊名CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management
出版商Association for Computing Machinery
5044-5048
页数5
ISBN(电子版)9781450392365
DOI
出版状态已出版 - 17 10月 2022
已对外发布
活动31st ACM International Conference on Information and Knowledge Management, CIKM 2022 - Atlanta, 美国
期限: 17 10月 202221 10月 2022

出版系列

姓名International Conference on Information and Knowledge Management, Proceedings

会议

会议31st ACM International Conference on Information and Knowledge Management, CIKM 2022
国家/地区美国
Atlanta
时期17/10/2221/10/22

指纹

探究 'An In-depth Interactive and Visualized Platform for Evaluating and Analyzing MRC Models' 的科研主题。它们共同构成独一无二的指纹。

引用此

Wu, Z., Fang, J., Xu, H., & Gao, K. (2022). An In-depth Interactive and Visualized Platform for Evaluating and Analyzing MRC Models. 在 CIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management (页码 5044-5048). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3511808.3557167