← Back to Test-Time Training

AdamW TTT

Test-Time Training
Used in
8 PRs
Best BPB
0.8265
Avg BPB
1.0478

Submissions

PR #430by sahiee-dev
1.1428
PR #462by JoeProAI
1.0672
PR #489by sofiabod
1.1327
PR #532by NotADevIAmaMeatPopsicle
1.0487
PR #555by ymrohit
1.0916
PR #1350by resouer
1.0046
PR #1485by ndokutovich
1.0679
PR #1488by ndokutovich
0.8265

Hyperparameters Across PRs

pr_numberparameters
430{"epochs":3,"learning_rate":0.001,"betas":[0.9,0.999],"frozen_layers":6}
462{"learning_rate":0.0005,"epochs":10,"weight_decay":0}
489{"learning_rate":0.0005,"weight_decay":0,"epochs":5}
532{"epochs":10,"learning_rate":0.001,"grad_clip":1,"all_params_unfrozen":true}
555{"epochs":10}
1350{"epochs":6,"freeze_first_blocks":2}
1485{"epochs":6,"learning_rate":0.0005,"freeze_blocks":2,"schedule":"cosine decay","pre_quant":true}
1488{"epochs":10,"learning_rate":0.00045,"freeze_blocks":1}