Researchers have developed ShamAN-Q, a new sub-1-bit post-training quantization method for large language models, which improves upon NanoQuant by incorporating a dense curvature metric and Mahalanobis reconstruction loss.
a sub-1-bit post-training quantization method
We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tract…
Abstract: We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry wi…
