← Back to Regularization
dropout
RegularizationUsed in
11 PRs
Best BPB
1.0270
Avg BPB
1.3214
Submissions
PR #340by starfly-web
1.2182PR #345by anandks2006
1.8522PR #820by mtybadger
1.6252PR #1021by abaybektursun
1.3250PR #1491by wisebreadloaf
1.6924PR #1520by taka6745
1.0824PR #1520by taka6745
1.0824PR #1650by Jaredcastorena
1.4233PR #1822by Unwindology
1.1785PR #2032by anmarhindi
1.0293PR #2039by anmarhindi
1.0270Hyperparameters Across PRs
| pr_number | parameters |
|---|---|
| 340 | {"rate":0.1,"scope":"attention and MLP blocks"} |
| 345 | {"loop_dropout":true} |
| 820 | {"rate":0} |
| 1021 | {"rates":[0.3,0.05]} |
| 1491 | {"rate":0} |
| 1520 | {"type":"Norm-PCT-Dropout","top_l2_norm_row_fraction":0.01,"target":"FFN intermediate activations"} |
| 1520 | {"type":"skip gates","description":"sigmoid-gated U-Net skip connections"} |
| 1650 | — |
| 1822 | {"type":"stochastic depth","expected_value_scaling":true} |
| 2032 | {"stochastic_depth_max":0.02} |
| 2039 | {"stochastic_depth_max":0.02} |