Jeremy emphasizes the value of competition papers as a treasure trove of practical insights and techniques that often go unnoticed in mainstream research. He advocates for a broad approach to learning, suggesting that diversifying sources of knowledge can lead to a more accurate understanding of effective model training. The discussion highlights the importance of comparing models and the necessity of being grounded in practical applications rather than theoretical purity.