← Back to LR Schedule

cosine warmdown with linear warmup

LR Schedule
Used in
1 PRs
Best BPB
1.0944
Avg BPB
1.0944

Submissions

PR #644by Christopher-Lee-McClendon
1.0944

Hyperparameters Across PRs

pr_numberparameters
644{"warmup_steps":20,"peak_lr_steps":12000,"warmdown_steps":13000}