← Back to Quantization

mixed int6

Quantization
Used in
19 PRs
Best BPB
0.0972
Avg BPB
1.0655

Submissions

PR #135by unnir
1.1539
PR #174by Julz19
1.1537
PR #339by sheeki03
1.1364
PR #398by felipe-parodi
1.1213
PR #581by teddyoweh
1.0698
PR #649by pall23-mech
1.2073
PR #684by DeepReinforce
1.0574
PR #698by hesong0222-dev
1.1642
PR #811by quietsmile
0.4377
PR #922by greqone
0.0972
PR #993by aerosta
0.9631
PR #1052by demouo
1.1978
PR #1427by kjahan
1.2092
PR #1438by sabdulmajid
1.2029
PR #1465by sisegod
1.1381
PR #1569by abbudjoe
1.3576
PR #1664by zoharb157
1.0980
PR #1665by mrbese
1.3571
PR #1696by kings-crown
1.1224

Hyperparameters Across PRs

pr_numberbitsscope
1356MLP and attention weight matrices; FP16 passthrough for tied embeddings and last 2 layers' Key projections
1746large MLP and attention matrices
3396model weights
3986all
5816model weights
6496all
6846model weights
6986all
8116model weights
9226model
9936post-training mixed
10526artifact
14276model weights
14386mlp;attn;embed
14656embeddings
15696default export
16646all
16656MLP, attention, and Mamba projection weights
16966attention/MLP banks