Nathan discusses the often misunderstood concept of constitutional AI, highlighting its two-stage process that involves revising instructions based on principles and generating synthetic data. He emphasizes the distinction between constitutional AI and reinforcement learning from AI feedback (Rlaif), and how these methods can enhance preference labeling. Despite its initial buzz, Nathan admits that even he struggled to grasp its full implications until recently.
