← Back to Architecture

SwiGLU FFN

Architecture
Used in
2 PRs
Best BPB
1.0672
Avg BPB
1.0926

Submissions

PR #462by JoeProAI
1.0672
PR #505by JoeProAI
1.1181

Hyperparameters Across PRs

pr_numberparameters
462—
505{"hidden":1792}