Natural Language to Visualization by Neural Machine Translation

Yuyu Luo, Nan Tang, Guoliang Li*, Jiawei Tang, Chengliang Chai, Xuedi Qin

*此作品的通讯作者

科研成果: 期刊稿件文章同行评审

70 引用 (Scopus)

摘要

Supporting the translation from natural language (NL) query to visualization (NL2VIS) can simplify the creation of data visualizations because if successful, anyone can generate visualizations by their natural language from the tabular data. The state-of-the-art NL2VIS approaches (e.g., NL4DV and FlowSense) are based on semantic parsers and heuristic algorithms, which are not end-to-end and are not designed for supporting (possibly) complex data transformations. Deep neural network powered neural machine translation models have made great strides in many machine translation tasks, which suggests that they might be viable for NL2VIS as well. In this paper, we present ncNet, a Transformer-based sequence-to-sequence model for supporting NL2VIS, with several novel visualization-aware optimizations, including using attention-forcing to optimize the learning process, and visualization-aware rendering to produce better visualization results. To enhance the capability of machine to comprehend natural language queries, ncNet is also designed to take an optional chart template (e.g., a pie chart or a scatter plot) as an additional input, where the chart template will be served as a constraint to limit what could be visualized. We conducted both quantitative evaluation and user study, showing that ncNet achieves good accuracy in the nvBench benchmark and is easy-to-use.

源语言英语
页(从-至)217-226
页数10
期刊IEEE Transactions on Visualization and Computer Graphics
28
1
DOI
出版状态已出版 - 1 1月 2022
已对外发布

指纹

探究 'Natural Language to Visualization by Neural Machine Translation' 的科研主题。它们共同构成独一无二的指纹。

引用此

Luo, Y., Tang, N., Li, G., Tang, J., Chai, C., & Qin, X. (2022). Natural Language to Visualization by Neural Machine Translation. IEEE Transactions on Visualization and Computer Graphics, 28(1), 217-226. https://doi.org/10.1109/TVCG.2021.3114848