PR #1061
openCausal Oscillator LM: physics-native architecture (BPB 1.34)
by rolandnsharpView on GitHub
val_bpb
1.3379
Architecture
Hybrid
Optimizer
Muon
Artifact Size
~11.2MB
Training Techniques
Architecture
Hybrid
Causal Oscillator Language Model using damped harmonic oscillator transfer functions as the core computational primitive, with causal FFT convolution over oscillator impulse responses and attention on physics-enriched states.
parameters: {"oscillators":192,"layers":12,"heads":16}
Optimizer
Muon
weight_decay: null
momentum: null
other_params: {"used_for":"2D weight matrices"}
AdamW
weight_decay: null
momentum: null
other_params: {"used_for":"oscillator parameters and embeddings"}
Quantization
int8
bits: 8
scope: model
Compression
custom
level: null
LR Schedule
linear warmup + cosine decay
parameters: null
Novel Contributions
- Causal Oscillator Language Model based on damped harmonic oscillator dynamics
- Causal convolution via FFT over oscillator impulse responses
- Physics-native temporal context instead of positional embeddings
- Tokens drive a bank of 192 oscillators through learned impulse coupling
- Same architecture is claimed to transfer across text and audio/sequential wave signals
- Physically interpretable parameters such as frequency and damping ratio