Creativity and Reasoning
Francois discusses the interplay between creativity and interpolation, suggesting that both humans and models engage in a mix of pattern matching and reasoning. He emphasizes that larger models become more sample efficient due to their ability to leverage reusable building blocks, but warns that they can struggle with truly novel inputs. The conversation also explores the concept of program synthesis within models, proposing that different layers may serve distinct roles in processing and reasoning.In this clip
From this podcast
Dwarkesh Podcast
Francois Chollet - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution
Related Questions
Can GPT-4 be used for reasoning in the context of the episode Francois Chollet - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution and the clip Creativity and Reasoning?
Can GPT-4 be used for reasoning?
Why does Francois Chollet emphasize discrete program search in the episode It's Not About Scale, It's About Abstraction - Francois Chollet?