← Back to Architecture

MLP expansion

Architecture
Used in
5 PRs
Best BPB
1.1355
Avg BPB
1.1880

Submissions

PR #579by newjordan
1.1355
PR #592by Skytuhua
1.1476
PR #966by andrewmouldon
1.2162
PR #1315by andrewmouldon
1.2207
PR #1551by andrewmouldon
1.2199

Hyperparameters Across PRs

pr_numberparameters
579{"hidden_dim":2560,"activation":"relu-squared"}
592{"expansion_factor":3,"hidden_dim":1536}
966{"baseline":"2.00x","short_conv":"1.99x","moc":"1.93x"}
1315{"scale_vs_baseline":2.65}
1551{"baseline_expand":"2x","temporary_expand":"4x","effective_training_width":"8x"}