Pavan discusses the intriguing role of orthonormality in deep learning, particularly within disentangling autoencoders. By imposing orthonormal constraints on latent blocks, they observed a significant improvement in disentangling quality, aligning mathematical principles with practical implementation. This approach not only enhances performance but also respects the underlying physics of image formation.
