Roland discusses the limitations of LNM models when trained on short sequences, highlighting their struggles with generalization beyond their training data. He emphasizes the need for recurrence in future models to effectively tackle complex problems, suggesting that incorporating recurrent connections could enhance their performance. The evolution of these models is seen as a necessary step forward, building on the insights gained from current architectures.
