← Back to Architecture

LoRA

Architecture
Used in
9 PRs
Best BPB
0.9623
Avg BPB
1.3237

Submissions

PR #103by MatthewHRockwell
1.5000
PR #777by Robby955
0.9623
PR #1300by Ribin545
1.8184
PR #1341by himanshudongre
1.1000
PR #1385by korentomas
1.4465
PR #1684by camden-git
1.2554
PR #1765by renqianluo
1.0727
PR #1765by renqianluo
1.0727
PR #2115by Sacmaj
1.6853

Hyperparameters Across PRs

pr_numberparameters
103{"rank":4}
777{"rank":8}
1300{"rank":512,"scope":"q"}
1341{"rank":8,"targets":["Q","V"]}
1385{"rank":16}
1684{"rank":16}
1765{"rank":128,"alpha":96}
1765—
2115{"rank":16,"roles":"attn_c_v,attn_proj,mlp_fc,mlp_proj"}