pool archive signal about

© 2026 sonicwatersonicwater

archive signal rss api · mcp media kit privacy? keys

0:00 / 4:34

dropout

140 bpm7AD minor4:34Oct 26, 2026Δ 5 390 MSW-008.04experiment · generative untrained transformers

mixes well with

mixture of experts7A · 136 Recast7B · 138 Tuning the Tierce7B · 132 Siphonophore7A · 134 Equinox7A · 132 Polar Night7A · 134

from the experiment

generative untrained transformers

attention heads138 bpm · 11B4:38
xavier uniform118 bpm5:25
latent space134 bpm · 11B5:00
dropout140 bpm · 7A4:34
masked tokens100 bpm4:48
residual stream124 bpm · 12A4:07
exploding gradients86 bpm · 12A3:54
emergent abilities165 bpm · 12B4:40
vanishing gradient132 bpm · 9A4:50
grokking112 bpm · 4B5:25
mixture of experts136 bpm · 7A4:42
latent walk122 bpm · 12B4:11
backpropagation164 bpm · 4A2:51
positional encoding134 bpm · 9A6:10