← Back to LR Schedule

linear warmup + constant + cosine cooldown

LR Schedule
Used in
1 PRs
Best BPB
1.7510
Avg BPB
1.7510

Submissions

PR #126by Athenox14
1.7510

Hyperparameters Across PRs

pr_numberparameters
126{"warmup_steps":100,"cooldown_steps":2000,"final_lr_multiplier":0.1}