Michael Arbel

What do JEPA-style self-distillation dynamics actually learn — and why do they sometimes avoid collapse?

In our new work with @BasileTerv987 and Jean Ponce, we tackle this question.

What surprised us: These dynamics provably recover representations with nonlinear-CCA structure.

Paper link: arxiv.org/abs/2605.17671

The core results:

1- JEPA-style self-distillation reveals a provable nonlinear-CCA structure, but the same dynamics can also stabilize collapse.

2- PEIRA keeps the canonical-correlation structure while making collapsed equilibria unstable.

Come check out our Kernel Bilevel Optimization poster (#3005) tomorrow at 4:30 PM PST at #NeurIPS2025 in San Diego!

Most existing generalization results for bilevel optimization are limited to simple parametric models. What about more flexible models? Kernels offer a powerful starting point!

Great work led by @fares_elkhoury!

📍 Exhibit Hall C,D,E — #3005
📄 Paper: openreview.net/pdf?id=bMKUD39…

Want tabular foundation models that seamlessly handle arbitrary output dimensions?

Come see our EquiTabPFN poster (#5510) at 4:30 PST at #NeurIPS2025!

With amazing collaborators David Salinas and Frank Hutter.

PS: We’re looking for a master’s intern for follow-up research!

Reverting back the reviews and scores seems unnecessary, #ICLR2026. Many discussions were actually honest and fruitful. Why not let the (new) ACs judge by themselves of possible collusions?