Magnus emphasizes the importance of RNNs and LSTMs as foundational concepts leading to the development of transformers. He highlights the complexity of transformers and the necessity of understanding encoder-decoder networks and attention mechanisms. Despite the rise of transformers, he believes that RNNs and LSTMs will continue to play a significant role in various applications, including computer vision.
