Evaluating Model Inference
Nathan and Riley delve into the intricacies of model evaluation, discussing the concept of equal inference costs and the potential of self consistency as a reliable method. Riley highlights that running multiple samples can yield results comparable to more advanced models, suggesting a nuanced approach to understanding trade-offs in inference costs. They also touch on recent advancements, such as the plan search method, which further refines these evaluation techniques.In this clip
From this podcast
Interconnects Audio
Interviewing Riley Goodside on the science of prompting
Related Questions