Illia and Patrick discuss the evolution of AI, highlighting the intermittent progress in exponential growth. They delve into the challenges of reasoning and common sense in current AI models, emphasizing the need for fundamental advancements to achieve the next step function in AI development.