← Back to Architecture

Eigenweight

Architecture
Used in
10 PRs
Best BPB
1.3223
Avg BPB
1.3223

Submissions

PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223
PR #1589by nnm2602
1.3223

Hyperparameters Across PRs

pr_numberparameters
1589{"rank":64}
1589{"rank":128}
1589{"rank":256}
1589{"rank":128}
1589{"attn_rank":32,"mlp_rank":96}
1589{"attn_rank":96,"mlp_rank":32}
1589{"attn_rank":128,"mlp_rank":32}
1589{"dimension":1024,"rank":64}
1589{"dimension":768,"rank":48}
1589{"dimension":1024,"rank":32}