Non-Contrastive Learning
Ishan discusses the limitations of contrastive learning, emphasizing the importance of high-quality negative samples. He introduces non-contrastive methods like clustering and self-distillation, which focus on maximizing similarity between features from different networks without relying on negatives. The conversation highlights the challenge of preventing "collapse," where all images could end up being represented the same way, rendering the learning ineffective.In this clip
From this podcast
Lex Fridman Podcast
Ishan Misra: Self-Supervised Deep Learning in Computer Vision | Lex Fridman Podcast #206
Related Questions