← Back to Quantization

mixed int5/int6/int7

Quantization
Used in
5 PRs
Best BPB
1.0631
Avg BPB
1.1030

Submissions

PR #332by saml212
1.1320
PR #422by albertorkive
1.1396
PR #1126by AnirudhRahul
1.1091
PR #1962by chris-colinsky
1.0631
PR #2143by upascal
1.0713

Hyperparameters Across PRs

pr_numberbitsscope
3325all weights with gradient-guided per-tensor allocation
422—all
1126—weights
1962—matrix weights
2143—q/proj/mlp_proj, kv/mlp_fc, tok_emb