← Back to Architecture

MoE

Architecture
Used in
3 PRs
Best BPB
1.1180
Avg BPB
1.2510

Submissions

PR #480by imyesung
1.1456
PR #981by BurguerJohn
1.4893
PR #1451by davie2009kh
1.1180

Hyperparameters Across PRs

pr_numberparameters
480{"experts":2,"expert_multiplier":1.5}
981{"moe_layers":0,"total_layers":2}
1451—