← Back to Architecture

wider-shallower Transformer

Architecture
Used in
1 PRs
Best BPB
1.3043
Avg BPB
1.3043

Submissions

PR #185by dttdrv
1.3043

Hyperparameters Across PRs

pr_numberparameters
185{"layers":4,"dimensions":768,"heads":12,"kv_heads":4}