Model Compression Insights
Joseph discusses the challenges of training larger models and the successful application of quantization techniques developed by a team member. He highlights that their models were significantly larger than typical benchmarks, emphasizing the importance of exploring the trade-off space in model architecture. The conversation reveals that while they used standard methods, the interaction of these techniques led to promising results in both training and inference.In this clip
From this podcast
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Rethinking Model Size: Train Large, Then Compress with Joseph Gonzalez - #378
Related Questions
How does increasing model size affect performance in deep learning as discussed in the episode Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94 and the clip Introduction to Deep Double Descent?
How does the size of a neural network affect its performance in deep learning, as discussed in the episode Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94 and the clip Deep Double Descent?
How does the size of a neural network affect its performance in deep learning, as discussed in the episode Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94 and the clip Introduction to Deep Double Descent?