Gradient Dissent - A Machine Learning Podcast avatar

Dexa/Gradient Dissent - A Machine Learning Podcast

Learn more
Published Dec 22, 2022

Jerome Pesenti — Large Language Models, PyTorch, and Meta

Jerome Pesenti delves into the transformative power and challenges of large language models, highlighting AI's role in revolutionizing drug discovery, education, and AR/VR interfaces, alongside insights into PyTorch's evolution at Meta and the importance of addressing bias and scalability issues.
Episode Highlights
Gradient Dissent - A Machine Learning Podcast logo

Topics covered

Popular Clips

Questions from this episode

Episode Highlights

  • Model Progress

    shares his insights on the advancements in large language models, highlighting their ability to perform multiple tasks and the trend towards multimodality and self-supervision. He expresses skepticism about the term "AGI," emphasizing that intelligence is inherently not general and that the journey to human-level intelligence is far from complete. Jerome notes, "We're 1% to human intelligence. We made real progress. Right. But intelligence is so amazing that you still have a long way to go."

    Intelligence is so amazing that you still have a long way to go.

    ---

    Despite the progress, he acknowledges the challenges, such as the models making silly mistakes that humans wouldn't make 1 2.

       

    Control Challenges

    Controlling the outputs of large language models and addressing inherent biases are significant challenges, according to . He points out that while these models can perform impressive tasks, they often produce unexpected results and exhibit extreme biases, making them unreliable for production use. Jerome illustrates this with examples of biased outputs, stating, "If you type CEO, guess what you get. If you type assistant, guess what you get."

    If you type CEO, guess what you get. If you type assistant, guess what you get.

    ---

    He stresses the importance of making these models more controllable and reducing biases to ensure their safe and effective deployment 1 2.

       

    AI Impact

    At Meta, AI applications like recommender systems have proven to be highly impactful, particularly in advertisements and organic recommendations. notes that while AI has not drastically changed everyday life, its influence is significant in specific domains. He remarks, "The most successful application of AI so far has been advertisement."

    The most successful application of AI so far has been advertisement.

    ---

    In drug discovery, Jerome sees potential for AI to revolutionize the field, akin to a "Tesla revolution," by improving decision processes and reducing costs 3 4 5.

Related Episodes