713: Llama 2, Toolformer and BLOOM: Open-Source LLMs — with Meta's Dr. Thomas Scialom

Community Impact

The release of open-source large language models (LLMs) like Llama 2 has significantly impacted the research community and industry. Thomas Scialom highlights that before Llama 2, major LLMs were only accessible through APIs, limiting research and commercial applications 1. By open-sourcing Llama 2, Meta has enabled broader access and innovation, allowing companies to use it under a commercial license, provided they have fewer than 700 million users 1. This shift has fostered a more collaborative environment, as Scialom notes, "We will be nowhere without open sourcing. Think about BERT, transformers, and even GPT-1."

We will be nowhere without open sourcing. Think about BERT, transformers, and even GPT-1.

--- Thomas Scialom

However, open-sourcing also presents challenges, particularly in ensuring safety and responsible use. Scialom acknowledges the difficulty in controlling open-source models compared to API-restricted ones, emphasizing the importance of implementing robust safety measures 2.

   

Ethical Concerns

Ethical concerns are at the forefront of discussions around open-source LLMs like Llama 2. Jon Krohn and Thomas Scialom explore the potential risks of connecting LLMs to real-world applications, such as hardware interfaces, which could amplify their impact 3. Scialom stresses the seriousness of these concerns, noting that the ability of models to take actions in the world introduces new challenges and research questions 3.

There's real risks at another order of magnitude.

--- Thomas Scialom

The development of Llama 2 involved extensive ethical considerations, including red teaming exercises to simulate potential misuse scenarios 4. This unprecedented level of concern for ethics and responsible use underscores the importance of addressing these issues as LLMs continue to evolve.