← Back to Architecture

LeakyReLU² stack

Architecture
Used in
1 PRs
Best BPB
1.0781
Avg BPB
1.0781

Submissions

PR #672by andrewbaggio1
1.0781

Hyperparameters Across PRs

pr_numberparameters
672{"layers":11,"d_model":512,"gqa_heads":"8/4","mlp_multiplier":3,"bigram_hash":2048,"partial_rope_dims":16}