Twitxy Saved

IdePHICS lab

@idephics

"Information Learning & Physics" is an academic lab. in EPFL, Switzerland, focusing on computer science, machine learning & statistical physics. PI: @krzakalaF
Lausanne epfl.ch/labs/idephics/ Joined October 2020
69 Following 213 Followers 480 Posts

Latest posts 5

IdePHICS lab @idephics
Check out our lastest work on understanding deep learning! With @DandiYatin, @l_arnaboldi, Matteo Vilucchio and Hugo Tabanelli 👇👇👇
Krzakala Florent Verified account @KrzakalaF May 14
What if a theory of deep learning could be built from iterated kernel spectral methods?

Feature learning, advantage of depth, emergence of concepts, convnets filters.... and a new backprop-free algorithm too! We have it all!

Introducing Neural LoFi 🧵
arxiv.org/abs/2605.13612
View quoted post on X
IdePHICS lab @idephics
Really excited about this 🇨🇭/🇫🇷 collaboration with the folks at @LightOnIO If you’re building RAG systems, this is worth a read. SOTA under 150M parameters 📷👇
Antoine Chaffin Verified account @antoine_chaffin Feb 19
State-of-the-art ColBERT models are trained by applying knowledge distillation on top of dense pre-trained models
What if we run the whole pre-training in the multi-vector?
Introducing ColBERT-Zero, a model that sets a new SOTA on BEIR, using only public data
View quoted post on X
IdePHICS lab @idephics
Agreed! In fact we at idephics recently investigated some of the marvels achieved by giant step size in learning complex functions with a neural net:

arxiv.org/abs/2305.18270
I think the reason why second-order methods keep underperforming relative to first-order methods in deep learning is that first-order methods are more powerful than the theory gives them credit for. First-order methods + large step sizes can implicitly access specific **third
View quoted post on X
IdePHICS lab @idephics
10 years ago, we started to apply statistical physics to signal processing, compressed sensing & learning sparsity. It started a chain of events that led to the creation of IdePHICS lab in EPFL!

Thanks to @PhysRevX for selecting the paper & to Dave Donoho for his kind words!
Physics Magazine @PhysicsMagazine Nov 17, 2021
The journal @PhysRevX was launched 10 years ago. From image processing to supersolids, we've asked experts to reflect on some pivotal PRX papers published over the last 10 years. physics.aps.org/articles/v14/1…
View quoted post on X