Search Results for author:
Found 3 papers, 0 papers with code
Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs
We introduce Integer Scale, a novel post-training quantization scheme for large language models that effectively resolves the inference bottleneck in current fine-grained quantization approaches while maintaining similar accuracies.
A Speed Odyssey for Deployable Quantization of LLMs
The large language model era urges faster and less costly inference.
FPTQ: Fine-grained Post-Training Quantization for Large Language Models
In this study, we propose a novel W4A8 post-training quantization method for the available open-sourced LLMs, which combines the advantages of both two recipes.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
