Published Jan 29, 2024
Model merging lessons in The Waifu Research Department
Nathan Lambert dives into the intricacies and advancements of model merging, discussing techniques like weight averaging, the challenges and practical benefits of these methods, and their mainstream and niche applications in machine learning.
Topics covered
Popular Clips
Episode Highlights
Related Episodes
A recipe for frontier model post-training
Answers 383 questionsWhy reward models are still key to understanding alignment
Answers 383 questionsText-to-video AI is already abundant
Answers 383 questionsHow to cultivate a high-signal AI feed
Answers 383 questionsSwitched to Claude 3.5
Answers 383 questionsFrontiers in synthetic data
Answers 383 questionsGoogle ships it: Gemma open LLMs and Gemini backlash
Answers 383 questionsOLMoE and the hidden simplicity in training better foundation models
Answers 383 questionsA realistic path to robotic foundation models
Answers 383 questionsA post-training approach to AI regulation with Model Specs
Answers 383 questionsDBRX: The new best open LLM and Databricks' ML strategy
Answers 383 questionsLocal LLMs, some facts some fiction
Answers 383 questions