PR #1793
open[Non-Record] Procedural Scaffolds for RMLA (1.5782 BPB RTX4090)
by sunburnt716View on GitHub
val_bpb
1.5782
Architecture
Transformer
Optimizer
Adam
Artifact Size
15.2 MB
Training Techniques
Test-Time Training
LoRA TTT
parameters: {"rank":8,"learning_rate":0.01}
Evaluation
stride-based eval
parameters: {"chunk_size":256,"eval_seq_len":1024,"batch_size":64}
Sequence Length
sequence_length
train_length: null
eval_length: 1024
Regularization
weight decay
parameters: null
Novel Contributions
- Per-document LoRA test-time training during evaluation
- Document-isolated evaluation with LoRA reset between documents
- Strided chunked evaluation over validation documents
- Batched document-wise adaptation for faster test-time training