Peter shares how OpenAI transitioned from Tensorflow to Pytorch organically, driven by the efficiency and readability of Pytorch code. The switch was facilitated during project transitions, avoiding disruptions mid-project. The move to Pytorch improved team productivity and tooling across the company.