← Back to Quantization

mixed int6/int8 with GPTQ-lite

Quantization
Used in
1 PRs
Best BPB
1.1804
Avg BPB
1.1804

Submissions

PR #543by rarce
1.1804

Hyperparameters Across PRs

pr_numberbitsscope
543—layers 1-9 int6, layers 0 and 10 int8, FP16 embeddings