Explainable AI model for predicting customer service communication effectiveness: A computational design science approach

Effective communication between customer service representatives (CSRs) and customers is essential for maintaining customer relationships and improving operational performance. Although firms have increasingly used AI technologies to evaluate customer service communication effectiveness, existing approaches primarily rely on dialogue texts while ignoring sequential communication strategies that reveal CSRs' underlying intentions. Furthermore, communication strategies are often quantified through manual coding, limiting scalability and explainability. Consequently, it remains unclear how individual strategies and their combinations contribute to AI-based predictions of communication effectiveness. Following the Computational Design Science research paradigm and Aristotle's Rhetoric Theory, this study proposes an explainable customer service communication effectiveness prediction model with text-strategy dual input based on the Transformer Encoder architecture (ExTSdiT). To enable automated strategy quantification, we design few-shot Chain-of-Thought prompts to guide large language models in annotating communication strategy labels from CSR utterances. Customized model explainability techniques are further employed to analyze the effects of individual strategies and their combinations at the dialogue-turn level. The framework is evaluated on a large-scale real-world customer service call dataset. Results show that the proposed model outperforms benchmark methods across multiple metrics and exhibits robust performance in ablation and stability analyses. The explainability results can support customer service communication quality inspection, and facilitate data-driven customer service training.