Jacob reflects on the unexpected advancements in language models like GPT-2 and GPT-3, acknowledging their surprising ability to generate coherent text. He discusses the evolution of NLP tasks, emphasizing that earlier models struggled with understanding the world rather than just the rules of language. This shift in perspective highlights the complexities of language understanding in artificial intelligence.
