Published Mar 21, 2024
Evaluations: Trust, performance, and price (bonus, announcing RewardBench)
Nathan Lambert delves into the intricacies of AI evaluations, balancing trust with performance, while addressing the rising costs that challenge non-tech giants and the need for governmental support. He introduces RewardBench and WildBench, innovative tools revolutionizing AI model assessments by closing critical gaps and overcoming biases.
Topics covered
Popular Clips
Episode Highlights
Related Episodes
Big Tech's LLM evals are just marketing
Answers 383 questionsWhy reward models are still key to understanding alignment
Answers 383 questionsAlignment-as-a-Service: Scale AI vs. the new guys
Answers 383 questionsThe end of the "best open LLM"
Answers 383 questionsHow to cultivate a high-signal AI feed
Answers 383 questions
Nous Hermes 3 and exploiting underspecified evaluations
Answers 383 questionsIt's 2024 and they just want to learn
Answers 383 questionsLlama 3: Scaling open LLMs to AGI
Answers 383 questionsA recipe for frontier model post-training
Answers 383 questionsA realistic path to robotic foundation models
Answers 383 questionsOpen Language Models (OLMos) and the LLM landscape
Answers 383 questionsModel merging lessons in The Waifu Research Department
Answers 383 questionsText-to-video AI is already abundant
Answers 383 questions