Keith and Alex discuss the challenges of handling 3D transformations in deep learning frameworks. They explore the limitations of vectors and the potential of using pose matrices to capture object-wise invariance. The conversation delves into the complexities of degrees of freedom and the need for further implementation and research in this area.