PR #1238

open

Non-record: TurboQuant mixed-precision int4/int5 (val_bpb=1.1521)

val_bpb

1.1521

Architecture

Transformer

Optimizer

—

Artifact Size

13.4 MB

Training Techniques

Quantization

mixed int4/int5

bits: null

scope: Q/K int5, V/O and MLP int4 in middle layers; boundary layers int5

Test-Time Training

score-first TTT

parameters: null

Role-based mixed-precision weight quantization using TurboQuant-guided layer sensitivity
Keeping Q/K projections at int5 while quantizing V/O and MLP weights to int4 in middle layers
Using int5 for boundary layers to preserve quality
Negative result showing int3 weight quantization is unusable for this model
Observation that weight quantization sensitivity differs from KV cache activation sensitivity