Program Aided Language Models

COCOGEN Methodology

The COCOGEN methodology offers a novel approach to structured common sense generation by representing graphs as Python classes. Aman Madaan explains that instead of breaking down graphs into sequences of edges, COCOGEN uses Python classes to represent the structure, making it easier to manage dependencies between events 1. This method allows for a more natural representation of tasks, such as baking a cake, by organizing them into a directed acyclic graph with nodes representing each step.

What we proposed is instead of doing all that, like writing the graph as a string, writing it as sequence of edges, why don't we represent the graph as a Python class?

--- Aman Madaan

Shuyan Zhou adds that while tweaking prompts can improve performance, COCOGEN's format of programming is more crucial than specific prompt adjustments 2.

   

Influence on PAL

COCOGEN's innovative approach significantly influenced the development of Program Aided Language Models (PAL). Aman Madaan describes PAL as a method that separates the task of understanding what needs to be done from how to do it, leveraging Python programs to solve arithmetic problems 3. This approach allows language models to generate a plan without performing calculations, which are instead executed by a Python runtime.

You don't really have to force the language model to do everything, you can only make it generate the plan on how to solve the problem.

--- Aman Madaan

Shuyan Zhou notes that PAL outperforms other prompting mechanisms by using intermediate reasoning steps, demonstrating its effectiveness across various datasets 3.