Reasoning in Language Models

Laura discusses her research on the relationship between procedural knowledge in pre-training and reasoning capabilities in large language models. She explores whether increased scale and data lead to genuine learning or merely enhanced memorization. The conversation delves into the challenges of evaluating modern models, especially as training data blurs the lines between test and training sets.