GPTQ vs AWQ on SageMaker: Faster AI Inference

Two formats, two bets about which weights matter. GPTQ compresses a trained model to 4 bits in a single pass. It quantizes one weight column at a time, then uses second-order sensitivity data to spread that rounding error across the weights it hasn't touched yet. AWQ goes another way. It finds the…