What Is AWQ in LLM Quantization and How It Works

AWQ stands for activation-aware weight quantization. It scales the most influential weight channels based on offline activation statistics, then quantizes everything else to ultra-low bit widths. By protecting a small fraction of salient weights, it keeps quantization error down without…