Diogo discusses the concept of overfitted data sets, emphasizing that recognition of specific datasets can indicate a lack of generalizability in solutions. He highlights the challenges faced by practitioners, such as the inconsistent performance of architectures like inception and resnets across different tasks. The conversation also touches on the rapid evolution of the field, suggesting that the complexities of deep learning might be more of a miracle than a problem.
