← Back to Architecture

EngramLite

Architecture
Used in
2 PRs
Best BPB
1.1091
Avg BPB
1.1119

Submissions

PR #1122by icryo
1.1146
PR #1126by AnirudhRahul
1.1091

Hyperparameters Across PRs

pr_numberparameters
1122{"buckets":8192,"heads":2,"orders":2,"dim_per_head":32}
1126{"heads":2,"orders":2,"buckets":8192}