← Back to LR Schedule

linear warmup

LR Schedule
Used in
14 PRs
Best BPB
1.0587
Avg BPB
1.3378

Submissions

PR #46by vavo
1.2697
PR #536by jaksenc
1.5140
PR #896by MVPandey
1.1896
PR #1125by jainpranjal97
1.1946
PR #1163by intelligentiaomni
1.5645
PR #1203by raider99k
1.3557
PR #1239by tmancino
1.5918
PR #1691by AVINASH0052
1.2244
PR #1842by Azura-Whiston
1.8838
PR #1866by deborahnelson8788726
1.5042
PR #1876by Meirzhan05
1.0801
PR #1880by Meirzhan05
1.0775
PR #1891by peytontolbert
1.2205
PR #1919by dev-pratap-singh
1.0587

Hyperparameters Across PRs

pr_numberparameters
46{"warmup_steps":100}
536{"warmup_steps":30}
896{"warmup_steps":200}
1125{"warmup_steps":20}
1163{"warmup_steps":20}
1203{"warmup_steps":5}
1239{"warmup_fraction":0.2}
1691{"warmup_steps":20}
1842{"warmup_steps":500}
1866{"start_step":0,"end_step":500}
1876{"steps":20}
1880{"warmup_steps":20}
1891{"warmup_steps":1}
1919{"warmup_chunks":100}