Meta releases its biggest ‘open’ AI model yet

Specifications

Meta's latest AI model, Llama 3.1, marks a significant milestone with its 405 billion parameters, making it the largest model the company has released. Jaeden Schafer highlights the model's open-source nature, although there are debates about its true openness due to certain regulations and stipulations 1. The model's capabilities are comparable to other leading AI models like GPT-4 and Claude 3.5, showcasing Meta's advancements in AI technology.

This is technically their largest model to date they've released. So it's called Llama 3.1. It's an open source AI model.

--- Jaeden Schafer

The release of Llama 3.1 has generated considerable excitement and hype, partly fueled by Mark Zuckerberg's promotional efforts.

   

Open Source Debate

The open-source label of Llama 3.1 has sparked controversy, as some argue it doesn't fully meet open-source criteria due to existing restrictions. Jaeden Schafer points out that despite these debates, the model remains accessible for free, which has been a key selling point for Meta's AI initiatives 1. This approach has allowed Meta to position itself as a more open alternative to other AI giants, even if the model's openness is somewhat limited.

Some people argue if this is, you know, actually open source, but we're going to get into all of that in the podcast today.

--- Jaeden Schafer

The ongoing debate highlights the complexities of defining and implementing open-source principles in the AI industry.

   

Training

Meta's Llama 3.1 is not only notable for its size but also for its competitive training techniques, which utilize 16,000 Nvidia H100 GPUs. Jaeden Schafer explains that such extensive resources are typically available only to major companies like Meta, enabling them to perform large-scale training runs 1. These advanced techniques have made Llama 3.1 competitive with top-tier models from companies like OpenAI and Anthropic.

The model was trained using 16,000 Nvidia H 100 gpu's. Right? This is something that only a company like meta or some of these really big companies really have the budgets to do.

--- Jaeden Schafer

This level of competitiveness underscores Meta's commitment to advancing AI technology and maintaining its position in the industry.