EATER: Entropy-Aware Multi-bit Watermarking for Large Language Models

The extensive generative capabilities of Large Language Models (LLMs) are causing the potential to increase abuse and misuse of AI-generated contents, which makes annotation and detection be critical tasks. Embedding watermarks into models is considered an effective approach that is imperceptible to humans but can be easily detected using specific algorithms. However, compared to the image and audio domains, the text domain possesses relatively limited redundancy. Most existing watermarking methods struggle to be effectively applied in scenarios requiring substantial embedding space, including low-entropy generation tasks and when it is necessary to embed multi-bit watermarks for advanced traceability. To address this challenge, in this work we propose a novel entropy-aware multi-bit watermarking framework. Specifically, by developing an entropy-aware filtering module, we estimate context entropy in real time to filter out invalid watermark spaces. Simultaneously, we integrate a multi-bit position allocation strategy to efficiently utilize the remaining space, thereby enabling trustworthy traceability of multi-bit watermark information in diverse generation scenarios. Comprehensive experiments demonstrate that our scheme improves text quality across various generation scenarios while maintaining high multi-bit decoding accuracy, providing a practical solution for the trustworthy traceability of LLMs.