← Back to LR Schedule

warmup + warmdown cosine schedule

LR Schedule
Used in
1 PRs
Best BPB
1.0865
Avg BPB
1.0865

Submissions

PR #548by LoquiAuris
1.0865

Hyperparameters Across PRs

pr_numberparameters
548{"warmup_steps":20,"warmdown_iterations":3000}