Scribe's Independent Research #44: Does the GRU use 'remembered' information for prediction? - Separating memory and generative judgment in StepMania chart prediction models

In sequence prediction using neural networks, the ability to retain past information in an internal state is crucial for handling long-term dependencies. In StepMania chart generation as well, the nex ...

[Thinking about Local LLMs Day 40] Fragility was visible without ablation — AUC 0.84 with baseline logit difference alone, 0.66 correlation with required unit count

The predictions of small models are on much thinner ice than you might think. When measured on day 39, the median difference ...