LLMs and 2D Reasoning
The discussion reveals surprising insights into the computational capabilities of LLMs, challenging the notion that they operate solely in a 1D text space. Despite initial assumptions, LLMs can infer 2D structures without explicit guidance, demonstrating remarkable generalization across different grid sizes. Experiments showed that additional positional encoding often offered little to no advantage, emphasizing the models' inherent strengths.In this clip
From this podcast
Machine Learning Street Talk (MLST)
Daniel Franzen & Jan Disselhoff - ARC Prize 2024 winners
Related Questions