← Back to Architecture

GQA / KV head count

Architecture
Used in
1 PRs
Best BPB
1.1472
Avg BPB
1.1472

Submissions

PR #179by devin-cog
1.1472

Hyperparameters Across PRs

pr_numberparameters
179{"layers":11,"num_heads":8,"num_kv_heads":4}