Episode 517: Jordan Adler on Code Generators

Tools & Techniques

Code transformation involves using various tools and techniques to convert code from one form to another. Jordan Adler explains that the process typically involves parsing code into a data structure, manipulating it, and then converting it back to code 1. Tools like Yellow Code help parse languages and represent programming code as data objects, such as abstract syntax trees, which are crucial for this transformation 1. However, simpler methods like regex can also be used, though they may increase error risk due to less context awareness 2.

You can just use regex or even kind of look for strings and kind of manipulate strings. Really any way that you can kind of manage string or text as strings you can use for code too.

--- Jordan Adler

Felienne Hermans highlights the importance of understanding the limitations and potential pitfalls of these simpler methods 2.

   

Transformation Challenges

Code transformation projects, especially those involving large codebases, present significant challenges. Jordan Adler notes that not all code representations can be easily converted back into source code, which complicates the transformation process 3. Additionally, the uniqueness of each codebase means that general transformation guidelines may not apply universally, leading to potential bugs 3.

You have to remember that when you're reading documentation about code transformation of some kind, you will find instructions or guidance is generally true, but may not be true for your specific case.

--- Jordan Adler

Felienne Hermans emphasizes the need for careful testing and consideration of edge cases to avoid errors during transformation 2.

   

Large Codebases

Handling large codebases during transformation requires strategic planning and execution. Jordan Adler shares insights from a multi-year migration project at Pinterest, where chunking the code into smaller parts helped manage the transformation process efficiently 4. This approach allowed for parallel processing and reduced manual effort, ultimately speeding up the project 4.

We chunked it up. So we ran ten pieces of ten files at a time, for example, out of thousands of files.

--- Jordan Adler

Felienne Hermans adds that maintaining code readability and consistency is crucial, as transformed code will continue to be used and maintained by developers 5.