← Back to Weight Averaging

SWA + EMA

Weight Averaging
Used in
2 PRs
Best BPB
1.1186
Avg BPB
1.1186

Submissions

PR #690by EthanYangTW
1.1186
PR #693by EthanYangTW
1.1186

Hyperparameters Across PRs

pr_numberparameters
690{"blend_ratio":"50/50","ema_decay":0.997,"swa_interval_steps":50}
693{"blend":"50/50","ema_decay":0.997,"swa_interval_steps":50}