Which Programming Language is ChatGPT Best At

Efficiency

Large language models (LLMs) are revolutionizing programming by enhancing efficiency and productivity. Kyle Polich shares how these models can significantly speed up coding tasks, allowing programmers to focus on more complex problems rather than repetitive tasks 1. Alessio Buscemi notes that while the idea of becoming a "ten x engineer" through LLMs is exaggerated, these tools still offer substantial improvements in productivity 2. He emphasizes the importance of understanding and controlling the code generated by these models to ensure quality and security.

You need to understand what it does, you need to be able to interpret the code that it writes, also to spot bugs and vulnerabilities.

--- Alessio Buscemi

The discussion highlights the potential of LLMs to transform software engineering by making routine tasks more efficient while still requiring human oversight.

   

Performance

The performance of LLMs in code generation varies significantly across different programming languages. Alessio Buscemi reveals that Python and Julia perform exceptionally well, with Julia surprisingly outperforming Python in some cases, despite its lower popularity 3. This raises questions about the correlation between language popularity and LLM performance. The discrepancies in performance are notable, with Python achieving nearly 100% success in generating correct code for algorithms, while languages like Smalltalk and C lag significantly behind 4.

If we challenge a GPT on the ten algorithms for Python, we reach nearly 100% of correct code being generated.

--- Alessio Buscemi

Buscemi suggests that the complexity of lower-level languages like C increases the difficulty for LLMs, potentially leading to a shift towards higher-level languages optimized for code generation 5.