LLMs and Video Analysis

LLMs show promise in scene description but struggle with structured outputs necessary for behavioral analysis. When tasked with identifying interactions within videos, they often miss key details or generate inaccurate information. Current methods typically analyze video as a series of independent frames, which may overlook crucial temporal context needed for understanding behavior.