← Back to Architecture

sliding window attention

Architecture
Used in
4 PRs
Best BPB
1.0205
Avg BPB
1.1273

Submissions

PR #5by albertorkive
1.2244
PR #914by mkenney2
1.1873
PR #1562by joshkmartinez
1.0205
PR #1728by mikeapedia
1.0771

Hyperparameters Across PRs

pr_numberparameters
5{"window_size":null}
914{"window_size":512}
1562—
1728{"local_window_512_layers":[0,1,2,3,5],"local_window_1024_layers":[6,7,8]}