PR #1461

open

Record Submission: HDC_1_Step_Grad_DSV_Radial_Slyvester_Hadamard_Matrix_Symmetry_Language_Model_val_bpb: 0.4118

val_bpb

0.4118

Architecture

Hybrid

Optimizer

—

Artifact Size

15,943,520 bytes

Training Techniques

Architecture

BigramHash

Hash-based bucketed next-token prediction pipeline with fingerprint routing and bucket frequency tables.

parameters: {"table_bits":19,"embed_dim":16}

DirectionalSemanticVec

Directional semantic vector layer using forward/backward semantic bundles and skip-bigram lags as the primary predictive signal.

parameters: {"lags":[2,3,4,5]}

Hadamard Matrix

Uses a Sylvester Hadamard matrix in the learning / codebook pipeline.

parameters: null

weight tying

Codebook / embedding-style shared representation is implied by the compact factorized pipeline.

parameters: null

Compression

lzma

level: 9

Regularization

magnitude pruning

parameters: {"threshold":1}

Other

other

1-step random gradient learning / single-iteration NMF-style update used to fit the hash-gradient factors.

parameters: {"nmf_max_iter":1}

Evaluation

multi-seed evaluation

parameters: {"seeds":[42,7,1337],"runs":3}

Sequence Length

sequence_length

train_length: null

eval_length: null