Mathilde discusses the challenges of training stability in machine learning models, particularly with the final classification layer. By replacing this layer with centroids from Kmin clustering, they significantly improved stability and performance. They also explored the idea of using a linear layer to evolve prototypes during training, drawing inspiration from techniques like Simclr to achieve online clustering with batch-level learning signals.
