@maxseitzer
Pinned post
📢 Save the date! Join us for the next @ELLISforEurope x @unireps Speaker Series!
📅8th October – 16:00 CEST
📍 ethz.zoom.us/j/66426188160
🎙️Speakers: Keynote Talk by @maxseitzer & Flash Talk by @JRaugelView quoted post on X
DINO-v3 has a single high-magnitude channel on its residual pathway, channel 416. Turning off this single channel affects DINO's entire output by 50-80%. For context, turning off a random channel has an effect of less than one percent.
The model builds up channel 416 in its last two layer-scale operations, using a single high-magnitude weight in each op to drastically ramp up channel 416's magnitude. This channel doesn't depend on the input, every image fires with a constant overlay.
After bringing channel 416 up to a value of about ten thousand, DINO-v3 then scales it down in the final layer-norm to almost nothing, removing it without a trace.View quoted post on X
arxiv.org DINOv3 Self-supervised learning holds the promise of eliminating the need for manual data annotation, enabling models to scale effortlessly to massive datasets and larger architectures. By not being... Introducing DINOv3 🦕🦕🦕
A SotA-enabling vision foundation model, trained with pure self-supervised learning (SSL) at scale.
High quality dense features, combining unprecedented semantic and geometric scene understanding.
Three reasons why this matters…View quoted post on X
