MG-CTG: A Framework for Controllable Text Generation Across Multiple Granularities

Managing text data is crucial given the abundance of unstructured textual data in real-world applications. Text generation not only assists in managing massive amounts of text through tasks such as summarization and report generation but also has the capability to generate the needed content to enrich the textual database. However, the generated text is often open-ended and may not meet specific target requirements that fall into three categories: semantic, structural, and lexical. Fine-tuning pre-trained language models can meet each specific control requirement, but there is no simultaneous integration of controls from all three categories. On the other hand, post-processing methods are limited to semantic control or lexical control only. In this paper, we propose MG-CTG, a Muti-Granularity Controllable Text Generation framework to generated text satisfying controls across multiple granularities. Specifically, we design distinct controllers that employ different strategies based on post-processing methods to achieve control. Further, our proposed framework is able to attain fine-grained control at the structural granularity, as well as enhance the incorporation of keywords into the generated text via a designed keyword-guided weighted decoding method. We conduct experiments by combining control information from different granularities and evaluate the results on standard benchmark dataset for controllable text generation. The experimental results demonstrate that our method outperforms other post-processing methods on two real-world datasets.