← Back to LR Schedule

warmup + warmdown

LR Schedule
Used in
12 PRs
Best BPB
0.6364
Avg BPB
1.1288

Submissions

PR #160by ChaseWNorton
1.1623
PR #164by jfprincz
1.1524
PR #254by timowhite88
1.1303
PR #256by IvGolovach
1.1779
PR #287by jfprinczRECORD
1.1271
PR #305by Naazimsnh02
1.1672
PR #400by chanwoo-park-official
1.1296
PR #451by harborglowvintage-oss
1.1464
PR #665by harborglowvintage-oss
1.1464
PR #746by C0neF
1.3556
PR #808by Naazimsnh02
0.6364
PR #858by nickferrantelive
1.2135

Hyperparameters Across PRs

pr_numberparameters
160{"warmup_steps":20,"warmdown_iters":3000}
164{"warmup_steps":1500,"warmdown_steps":3000}
254{"warmup_steps":1500,"warmdown_steps":3000}
256{"warmup_steps":20,"warmdown_iters":3000}
287{"warmup_steps":1500,"warmdown_steps":3000}
305{"warmup_steps":20,"warmdown_iters":3000}
400{"warmup_steps":20,"warmdown_iters":3000}
451{"warmup_steps":20,"warmdown_iters":3000}
665{"warmup_steps":20,"warmdown_iters":3000}
746{"warmup_steps":20,"warmdown_iters":1200}
808{"warmup_steps":1500,"warmdown_iters":3500}
858{"warmup_steps":20,"warmdown_iterations":1200}