← Back to Quantization

int5/int6

Quantization
Used in
3 PRs
Best BPB
1.1417
Avg BPB
1.1659

Submissions

PR #511by AnirudhRahul
1.1417
PR #515by keshav55
1.1807
PR #547by shajalahamedcse
1.1752

Hyperparameters Across PRs

pr_numberbitsscope
511——
515—MLP weights (int5), attention weights (int6, per-row scale)
547—MLP matrices (int5), attention matrices (int6), embeddings (int6)