← Back to Quantization

mixed int5/int6 GPTQ

Quantization
Used in
2 PRs
Best BPB
1.0903
Avg BPB
1.0916

Submissions

PR #1260by dexhunter
1.0929
PR #1817by Tonyy1977
1.0903

Hyperparameters Across PRs

pr_numberbitsscope
1260—all layers
1817—flat attention int5, rest int6, embeddings int8