July 1, 2026
April 11, 2026
Gabriel Yanci Arteaga, Marius Aasan, Rwiddhi Chakraborty, Martine Hjelkrem Tan, Thalles Silva, Michael Kampffmeyer, Adín Ramírez Rivera
Prototypical self-supervised learning methods consistently suffer from partial prototype collapse, where multiple prototypes converge to nearly identical representations. This undermines their central purpose—providing diverse and informative targets to guide encoders toward rich representations—and has led practitioners to over-parameterize prototype sets or add ad-hoc regularizers, which mitigate symptoms rather than address the root cause. We empirically trace the collapse to the joint optimization of encoders and prototypes, which encourages a type of shortcut learning: early in training prototypes drift toward redundant representations that minimize loss without necessarily enhancing representation diversity. To break the joint optimization, we introduce a fully decoupled training strategy that learns prototypes and encoders under separate objectives. Concretely, we model prototypes as a Gaussian mixture updated with an online EM-style procedure, independent of the encoder’s loss. This simple yet principled decoupling eliminates prototypecollapse without explicit regularization and yields consistently diverse prototypes, which in several settings translate to improved downstream performance.
Why Prototypes Collapse: Diagnosing and Preventing Partial Collapse in Prototypical Self-Supervised Learning
Gabriel Yanci Arteaga, Marius Aasan, Rwiddhi Chakraborty, Martine Hjelkrem Tan, Thalles Silva, Michael Kampffmeyer, Adín Ramírez Rivera
International Conference on Learning Representations (ICLR) 2026
April 11, 2026





Gabriel Yanci Arteaga, Marius Aasan, Rwiddhi Chakraborty, Martine Hjelkrem Tan, Thalles Silva, Michael Kampffmeyer, Adín Ramírez Rivera
International Conference on Learning Representations (ICLR) 2026
April 11, 2026




