702: Llama 2 — It's Time to Upgrade your Open-Source LLM — with Jon Krohn (@JonKrohnLearns)

Release

Meta's release of Llama 2 marks a significant milestone in the realm of open-source large language models. Jon Krohn highlights that Llama 2, unlike its predecessor, is open-source and commercially viable for applications with fewer than 700 million monthly active users 1. This strategic move limits its use by major competitors like Google. The model is pre-trained on 2 trillion tokens, a 40% increase from Llama 1, enhancing its capabilities significantly. Krohn notes:

Llama 2 is pre-trained on 2 trillion tokens of natural language data, which is 40% more than the original Llama 1.

--- Jon Krohn

Additionally, Llama 2 offers a chat variant fine-tuned on over a million human annotations, making it highly effective for chat applications 1.

   

Variants

Llama 2 introduces a diverse range of model variants, each tailored for specific applications. Jon Krohn explains that the model family includes both pre-trained and chat fine-tuned variants, available in sizes ranging from 7 billion to 70 billion parameters 2. The 70 billion parameter model, particularly the chat fine-tuned version, achieves performance comparable to ChatGPT on various benchmarks, setting a new standard for open-source models. Krohn states:

The 70 billion parameter Llama 2 model sets the new state of the art.

--- Jon Krohn

However, Llama 2's performance in tasks involving code or math remains a challenge, indicating areas for future improvement 2.