← Back to LR Schedule

warmup

LR Schedule
Used in
13 PRs
Best BPB
1.2073
Avg BPB
1.6075

Hyperparameters Across PRs

pr_numberparameters
56{"warmup_steps":10}
240{"warmup_steps":4}
248{"warmup_steps":4}
258{"warmup_steps":4}
276{"warmup_steps":4}
649{"warmup_steps":20}
1034{"warmup_steps":20}
1299
1300{"warmup_steps":12}
1434{"warmup_fraction":0.382}
1600{"warmup_steps":20}
1620{"warmup_steps":20}
1650{"warmup_steps":20}