Alekh discusses the limitations of supervised learning in interactive environments, using Atari games as a testing ground. He emphasizes the importance of adaptability, illustrating how a conversational agent could benefit from human intervention to learn from mistakes and improve future interactions. This highlights the need for algorithms to handle unexpected scenarios effectively.