Language Model Reasoning

Tim discusses the issue of language models memorizing data rather than reasoning. Andrew highlights concerns about arithmetic problems due to tokenization issues in language models. The Big Bench dataset challenges language models with tasks beyond simple language processing, revealing interesting patterns of results.