Kyle explains the concept of one shot learning, a class of algorithms that allows machine learning to recognize and learn from new and unusual data. He also discusses the difference between outliers and legitimate use cases for one shot learning, emphasizing the role of humans in developing techniques and methodologies for machine learning algorithms.