Chelsea discusses the significance of understanding not just the actions of a human but also their underlying intent in imitation learning. By inferring the reward function, a model can generalize its approach to new scenarios, rather than simply mimicking specific actions. This insight highlights the complexities of designing effective reward functions in real-world tasks, such as pouring water without spilling.