Published Dec 13, 2023
Big Tech's LLM evals are just marketing
Nathan Lambert exposes how big tech's marketing tactics skew language model evaluations, creating misleading scores and diluted public perception, while emphasizing the promise of in-context learning for cost-effective model personalization.
Topics covered
Popular Clips
Episode Highlights
Related Episodes
Open Language Models (OLMos) and the LLM landscape
Answers 383 questionsThe end of the "best open LLM"
Answers 383 questionsLocal LLMs, some facts some fiction
Answers 383 questionsEvaluations: Trust, performance, and price (bonus, announcing RewardBench)
Answers 383 questionsGoogle ships it: Gemma open LLMs and Gemini backlash
Answers 383 questionsThe koan of an open-source LLM
Answers 383 questionsLlama 3: Scaling open LLMs to AGI
Answers 383 questionsIt's 2024 and they just want to learn
Answers 383 questionsLlama 3.1 405b, Meta's AI strategy, and the new open frontier model ecosystem
Answers 383 questionsOLMoE and the hidden simplicity in training better foundation models
Answers 383 questionsWhere 2024’s “open GPT4” can’t match OpenAI’s
Answers 383 questions
A recipe for frontier model post-training
Answers 383 questions
Interviewing Ross Taylor on LLM reasoning, Llama fine-tuning, Galactica, agents
Answers 383 questions
