Richard Socher — The Challenges of Making ML Work in the Real World

Model Innovation

Controllable language models like CTRL offer a new dimension in natural language processing by allowing users to guide the output through control codes. Richard Socher explains that these models enable more precise generation by specifying the genre or task, such as translating text or continuing a story in a specific style 1. This approach contrasts with traditional models that generate text based on initial input without specific guidance. Socher highlights the transformative potential of these models:

It's amazing that that works. I mean, I can't believe that that works.

--- Richard Socher

The integration of control codes into large language models marks a significant step towards solving standard NLP problems with a single multitask model 2.

   

Future Directions

The future of NLP is poised for significant advancements through the use of controllable models. Richard Socher envisions a landscape where a single model can handle multiple tasks, moving beyond incremental improvements to a cumulative enhancement of capabilities 3. He suggests that the focus should shift from architecture engineering to refining objective functions, allowing for more efficient training of large neural networks. Socher shares his vision for the NLP community:

If we're able to do that and every research that we do actually makes an existing supermodel better and better, then we would all of a sudden have an explosion, I think, in progress in natural language processing.

--- Richard Socher

This shift could lead to a more unified approach in NLP, where each advancement builds upon the last, accelerating progress in the field.