← Back to Architecture

1+7+1 layer stack

Architecture
Used in
1 PRs
Best BPB
1.1194
Avg BPB
1.1194

Submissions

PR #625by Joeavaib
1.1194

Hyperparameters Across PRs

pr_numberparameters
625{"reasoning_layer":1,"completion_blocks":7,"validation_layer":1,"BigramHash_vocab_size":1536,"RoPE_dims":16}