← Back to Architecture

KV heads

Architecture
Used in
1 PRs
Best BPB
1.1634
Avg BPB
1.1634

Submissions

PR #552by loveless2001
1.1634

Hyperparameters Across PRs

pr_numberparameters
552{"kv_heads":4,"attention_heads":8}