← Back to LR Schedule

warmup + warmdown + cosine decay

LR Schedule
Used in
1 PRs
Best BPB
1.1532
Avg BPB
1.1532

Submissions

PR #456by Christopher-Lee-McClendon
1.1532

Hyperparameters Across PRs

pr_numberparameters
456{"warmup_steps":20,"warmdown_steps":3000,"total_steps":5200}