Pinned post
Curriculum learning changes the data. Progressive stacking changes the model. Human development changes both at once.
What happens when language model pretraining does the same?
We explore this in Curriculum-Guided Layer Scaling (CGLS) 🧵
What happens when language model pretraining does the same?
We explore this in Curriculum-Guided Layer Scaling (CGLS) 🧵