Acting Technique: Factors that Influence the Effectiveness of Role-Playing Model

The goal of role-playing is to ensure that LLMs generate responses that fully embody the specified character, adhering to the role’s background, speaking style, personality, and typical behaviors. Various methods aim to improve LLMs’ character simulation capabilities, such as leveraging specialized prompts and fine-tuning with character dialogues. However, no research has yet investigated the relationship between LLMs’ capabilities and their role-playing performance. Understanding this relationship could provide insights into selecting the most suitable base model for mimicking realistic characters and improving LLM performance from the outset of the task. In this paper, we investigate three main factors influencing role-playing performance: LLMs’ personalities and those of the target characters, LLMs’ familiarity with the character, and their conversational abilities. We conduct experiments with 11 models (including both prompt-based and fine-tuning methods) across 7 characters with varying personalities and backgrounds. The results indicate that both LLMs’ and characters’ personalities are primary factors affecting performance, while insufficient familiarity leads to poor outcomes. Importantly, we find no evidence that sharing the same personality with a character directly correlates with enhanced role-playing performance.