← Back to LR Schedule

WSD

LR Schedule
Used in
2 PRs
Best BPB
0.3212
Avg BPB
0.8321

Submissions

PR #850by callithyia
0.3212
PR #2094by Gitcatmeoww
1.3430

Hyperparameters Across PRs

pr_numberparameters
850{"stable_fraction":0.75,"decay":"cosine"}
2094{"stable_fraction":0.6,"decay_shape":"linear","min_lr_frac":0.1,"use_wsd":1}