← Back to Quantization

mixed int6/int5/int4/fp16

Quantization
Used in
1 PRs
Best BPB
1.1465
Avg BPB
1.1465

Submissions

PR #1456by sisegod
1.1465

Hyperparameters Across PRs

pr_numberbitsscope
1456—Q/K 6-bit, V/O 5-bit, MLP up ~2.3-bit, MLP down 4-bit, embeddings fp16