Mike explains the importance of the attention mechanism in capturing the structure of documents and how it outperforms naive approaches. He also discusses the contextual aspect of the attention mechanism and the benefits of training it jointly with the LSTM.
