跳到主要导航 跳到搜索 跳到主要内容

大语言模型低比特量化方法综述

  • Zhi Wei Hao
  • , Guan Yu Xu
  • , Li Shen
  • , Yong Luo
  • , Dan Zeng*
  • , Han Hu
  • *此作品的通讯作者
  • Beijing Institute of Technology
  • Sun Yat-Sen University
  • Wuhan University
  • Hangzhou Dianzi University
  • Shanghai University

科研成果: 期刊稿件 › 文献综述 › 同行评审

摘要

In recent years, Large Language Models (LLMs), most of which are built upon the Transformer architecture, have achieved remarkable success in a wide range of natural language processing tasks and have become a fundamental paradigm for intelligent systems. Their strong capabilities in language understanding, text generation, reasoning, instruction following, and interactive applications have greatly expanded the practical scope of artificial intelligence. However, these advantages are accompanied by rapidly growing model sizes and inference costs. As the number of parameters increases from billions to tens or even hundreds of billions, LLM deployment is increasingly constrained by memory capacity, memory bandwidth, computational overhead, and energy consumption. These issues pose a major obstacle to the efficient deployment of LLMs on consumer-grade devices, edge platforms, and other resource-constrained environments. To address this challenge, low-bit quantization has emerged as one of the most effective model compression techniques. By converting high-precision floating-point representations into lower-precision numerical formats, quantization can significantly reduce storage requirements, alleviate bandwidth pressure, lower power consumption, and improve inference efficiency, thereby enabling more practical deployment of LLMs. Although recent years have witnessed rapid progress in this area, existing studies are often scattered across different algorithmic settings, evaluation protocols, and deployment targets, and a systematic overview that connects fundamental concepts, methodological development, and engineering support is still relatively lacking. Therefore, this paper presents a comprehensive survey of low-bit quantization methods for LLMs. We first introduce the basic concepts of quantization, including low-precision numerical representations, quantization granularity, and the main factors that influence quantization error and model performance. On this basis, we organize existing methods into two major paradigms: Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). PTQ is emphasized as the dominant practical solution because it compresses pretrained models without expensive retraining, while QAT provides an important alternative by incorporating quantization simulation during training so that the model can better adapt to low-bit constraints. Within PTQ, this survey further reviews three major technical branches according to quantization targets and deployment goals. The first branch is weight-only quantization, which mainly focuses on compressing model parameters to reduce storage cost and memory traffic; representative directions include multi-bit and single-bit weight quantization, outlier handling, mixed-precision strategies, structured preprocessing, optimization-based quantization, and emerging non-uniform or ultra-low-bit methods. The second branch is joint weight-activation quantization, whose key objective is not only memory reduction but also actual low-precision computation acceleration during inference; related techniques include range migration, geometric transformation, activation-aware design, and low-bit computational optimization. The third branch is KV cache quantization, which is particularly important in autoregressive generation because the memory consumption of cached keys and values grows with sequence length and can become a major runtime bottleneck. In addition, this paper briefly reviews the software and hardware ecosystem supporting LLM quantization, including mainstream development frameworks, specialized optimization libraries, hardware accelerators, and deployment toolchains. Finally, we summarize current challenges and discuss future research directions, such as more robust ultra-low-bit quantization, better coordination between accuracy and efficiency, hardware–algorithm co-design, and more unified evaluation standards. By integrating methodological principles, representative techniques, and deployment considerations into a unified taxonomy, this survey aims to provide a clear reference for both academic researchers and industrial practitioners working on efficient LLM inference.

投稿的翻译标题A Survey of Low-Bit Quantization Methods for Large Language Models
源语言繁体中文
页(从-至)1936-1965
页数30
期刊Jisuanji Xuebao/Chinese Journal of Computers
卷49
期8
DOI
出版状态已出版 - 8月 2026
已对外发布是

关键词

  • deep learning
  • large language model
  • model compression
  • model inference
  • quantization

学术指纹

探究 '大语言模型低比特量化方法综述' 的科研主题。它们共同构成独一无二的学术指纹。

引用此