← Back to Test-Time Training

TTT

Test-Time Training
Used in
35 PRs
Best BPB
0.0214
Avg BPB
1.1111

Submissions

PR #196by sicauzxl
1.3825
PR #212by mrdavtan
1.1329
PR #367by ksang123
1.1770
PR #371by mrdavtan
1.1401
PR #588by andyluo22
1.4120
PR #645by FlynnCruse
1.8990
PR #646by Upsalla
1.1349
PR #651by phulin
1.2093
PR #687by RoyiRa
1.0745
PR #818by lucamignatti
0.5527
PR #901by Hilo-Hilo
1.1590
PR #962by AnirudhRahul
0.0214
PR #1026by danielxmed
1.0945
PR #1058by resouer
1.1247
PR #1184by icryo
0.9485
PR #1243by simon-marcus
1.1230
PR #1250by ibarrajo
1.2094
PR #1251by ibarrajo
1.1349
PR #1307by amrayach
1.1101
PR #1398by Mertyandimata
1.1047
PR #1414by Abhishek8108
0.7093
PR #1569by abbudjoe
1.3576
PR #1578by mikeapedia
1.0668
PR #1814by suryavanshi
1.0742
PR #1832by sricursion
1.0992
PR #1849by VedantKmr0
1.0850
PR #1867by lijuncheng16
1.7080
PR #1907by GodlyDonuts
1.1071
PR #1954by Syed-M-Zeeshan
1.0811
PR #1970by bsisduck
1.0674
PR #1973by harborglowvintage-oss
1.2193
PR #2009by SlavH
1.0500
PR #2013by Wilbatronic
1.0543
PR #2028by Arnie016
1.0898
PR #2080by NewyorkDev
0.9727

Hyperparameters Across PRs

pr_numberparameters
196{"run_ttt_eval":1}
212{"max_steps":500,"freeze_blocks":1}
367{"learning_rate":0.002}
371{"epochs":3,"optimizer":"SGD"}
588—
645—
646—
651—
687{"learning_rate":0.0001,"chunk_tokens":131072,"use_mixer":true}
818—
901—
962{"epochs":0,"freeze_blocks":2,"learning_rate":0.0001}
1026—
1058{"enabled":false}
1184{"enabled":false}
1243{"enabled":0}
1250—
1251—
1307{"enabled":false}
1398—
1414{"variant":"Discriminative TTT","per_block_adaptive_lr":true,"pre_quantization":true}
1569{"mode":"off"}
1578—
1814{"enabled":false}
1832{"enabled":true,"learning_rate":0.005,"epochs":3}
1849—
1867{"enabled":false}
1907{"phased":true}
1954{"learning_rate":0.006,"epochs":3}
1970{"beta2":0.999}
1973{"epochs":1,"learning_rate":0.005,"momentum":0.9,"chunk_size":32000}
2009{"mode":"backward-only","adaptation_target":"layer norms"}
2013{"rank":8}
2028{"enabled":false}
2080{"enabled":false}