Next talk tomo #ICML2026: How far does Taylor's Thm take us in deep learning?
@alex_damian_ and I take the local quadratic approx seriously, asking how well it actually holds during LLM pretraining.
Surprisingly, on 150M models, it tracks the true loss deep into training ↓
@alex_damian_ and I take the local quadratic approx seriously, asking how well it actually holds during LLM pretraining.
Surprisingly, on 150M models, it tracks the true loss deep into training ↓