Douwe discusses the evolution of language models, highlighting the intriguing phenomenon of "bertology" as a follow-up to earlier breakthroughs like word2vec. He emphasizes that current mathematical tools may not fully capture what models like BERT learn, suggesting that their performance might not reflect true linguistic understanding. The conversation also touches on the human tendency to anthropomorphize language-generating models, attributing them with more intelligence than they may actually possess.
