#046 The Great ML Stagnation (Mark Saroufim and Dr. Mathew Salvaris)

Metrics & Marketing

Mark Saroufim highlights how large numbers and metrics are often used as marketing tools in the tech industry, influencing perception and decision-making. He argues that while big numbers, such as trillions of parameters, may seem impressive, they often serve as a successful marketing channel rather than a true indicator of innovation 1. This approach is reinforced by the short tenure of employees at large tech companies, where quick promotions are prioritized over long-term thinking 1. Saroufim notes, "Look at what people say, look at what they do. And what they do seems to be working fine, so they're just going to keep doing it."

Look at what people say, look at what they do. And what they do seems to be working fine, so they're just going to keep doing it.

--- Mark Saroufim

The focus on metrics is mirrored in academia, where the number of papers published often dictates success, leading to incremental rather than groundbreaking work 2.

   

Incentive Structures

Incentive structures within tech companies and academia significantly shape research and development priorities. Saroufim explains that companies like Google and Microsoft finance incremental research because it reliably produces benefits, even if it doesn't lead to breakthroughs 3. This approach is driven by individual incentives, where employees are motivated by promotions and financial gain rather than risk-taking 3. He suggests that the current system creates a "local optimum" where safe, incremental work is favored over innovative risks.

If I do incremental research and it fails, no one's going to tell me, well, look, what you did here was really insane.

--- Mark Saroufim

In academia, the expectation has shifted from pursuing knowledge for its own sake to achieving financial success, which discourages risk-taking and innovation 4. Saroufim points out that while machine learning offers the potential for scientists to become wealthy, many prioritize financial stability over scientific discovery 4.