Abstract
In recent years, Large Language Models (LLMs), most of which are built upon the Transformer architecture, have achieved remarkable success in a wide range of natural language processing tasks and have become a fundamental paradigm for intelligent systems. Their strong capabilities in language understanding, text generation, reasoning, instruction following, and interactive applications have greatly expanded the practical scope of artificial intelligence. However, these advantages are accompanied by rapidly growing model sizes and inference costs. As the number of parameters increases from billions to tens or even hundreds of billions, LLM deployment is increasingly constrained by memory capacity, memory bandwidth, computational overhead, and energy consumption. These issues pose a major obstacle to the efficient deployment of LLMs on consumer-grade devices, edge platforms, and other resource-constrained environments. To address this challenge, low-bit quantization has emerged as one of the most effective model compression techniques. By converting high-precision floating-point representations into lower-precision numerical formats, quantization can significantly reduce storage requirements, alleviate bandwidth pressure, lower power consumption, and improve inference efficiency, thereby enabling more practical deployment of LLMs. Although recent years have witnessed rapid progress in this area, existing studies are often scattered across different algorithmic settings, evaluation protocols, and deployment targets, and a systematic overview that connects fundamental concepts, methodological development, and engineering support is still relatively lacking. Therefore, this paper presents a comprehensive survey of low-bit quantization methods for LLMs. We first introduce the basic concepts of quantization, including low-precision numerical representations, quantization granularity, and the main factors that influence quantization error and model performance. On this basis, we organize existing methods into two major paradigms: Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). PTQ is emphasized as the dominant practical solution because it compresses pretrained models without expensive retraining, while QAT provides an important alternative by incorporating quantization simulation during training so that the model can better adapt to low-bit constraints. Within PTQ, this survey further reviews three major technical branches according to quantization targets and deployment goals. The first branch is weight-only quantization, which mainly focuses on compressing model parameters to reduce storage cost and memory traffic; representative directions include multi-bit and single-bit weight quantization, outlier handling, mixed-precision strategies, structured preprocessing, optimization-based quantization, and emerging non-uniform or ultra-low-bit methods. The second branch is joint weight-activation quantization, whose key objective is not only memory reduction but also actual low-precision computation acceleration during inference; related techniques include range migration, geometric transformation, activation-aware design, and low-bit computational optimization. The third branch is KV cache quantization, which is particularly important in autoregressive generation because the memory consumption of cached keys and values grows with sequence length and can become a major runtime bottleneck. In addition, this paper briefly reviews the software and hardware ecosystem supporting LLM quantization, including mainstream development frameworks, specialized optimization libraries, hardware accelerators, and deployment toolchains. Finally, we summarize current challenges and discuss future research directions, such as more robust ultra-low-bit quantization, better coordination between accuracy and efficiency, hardware–algorithm co-design, and more unified evaluation standards. By integrating methodological principles, representative techniques, and deployment considerations into a unified taxonomy, this survey aims to provide a clear reference for both academic researchers and industrial practitioners working on efficient LLM inference.
| Translated title of the contribution | A Survey of Low-Bit Quantization Methods for Large Language Models |
|---|---|
| Original language | Chinese (Traditional) |
| Pages (from-to) | 1936-1965 |
| Number of pages | 30 |
| Journal | Jisuanji Xuebao/Chinese Journal of Computers |
| Volume | 49 |
| Issue number | 8 |
| DOIs | |
| Publication status | Published - Aug 2026 |
| Externally published | Yes |
Fingerprint
Dive into the research topics of 'A Survey of Low-Bit Quantization Methods for Large Language Models'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver