← Back to Quantization

mixed int4/int5

Quantization
Used in
3 PRs
Best BPB
1.1257
Avg BPB
1.1564

Submissions

PR #375by charmquark1984
1.1257
PR #1221by amabito
1.1915
PR #1238by ibarrajo
1.1521

Hyperparameters Across PRs

pr_numberbitsscope
375—MLP and attention
1221—all
1238—Q/K int5, V/O and MLP int4 in middle layers; boundary layers int5