Limits and Surprising Failures

Language models can struggle with basic arithmetic, such as multiplying large numbers. Despite being trained on a corpus, they may not have enough examples to accurately perform these calculations. However, there are ways to address these limitations and improve their performance.