Deep learning's remarkable ability to generalize across unfamiliar domains is highlighted, with examples like adapting imagenet models to cartoon images. The discussion emphasizes the vast amounts of underutilized data and the need for smarter approaches in unsupervised and multitask learning. Additionally, it points out critical gaps in current benchmarking practices, particularly regarding long-term dependencies and visual attention.
