Databricks Drops Open Source LLM DBRX, Beats Meta, xAI, Mixtral, GPT 3.5

Launch

Databricks has announced the launch of their open-source large language model, DBRX. Jaeden Schafer highlights that this model is designed to compete with leading AI models like GPT-3.5 and others. Ali Ghoshi, CEO of Databricks, expressed excitement about DBRX's potential to drive the industry towards more powerful and efficient open-source AI 1.

We're excited to share DBRX with the world and drive the industry towards more powerful and efficient open-source AI.

--- Ali Ghoshi

Jaeden notes that while foundational models like GPT-4 are great for general purposes, Databricks focuses on building custom models for each client that deeply understand their proprietary data 1.

   

Context

The launch of DBRX occurs within a highly competitive AI landscape. Jaeden mentions that despite significant investments by major companies, no one has yet matched the capabilities of GPT-4 1. Databricks' approach involves a mixture of experts architecture, similar to GPT-4, which enhances the model's efficiency and performance 2.

This seems like industry standards is what everyone's doing now. So, yeah, hats off to them.

--- Jaeden Schafer

Jaeden also points out that the development of DBRX took about two months and cost $10 million, showcasing Databricks' rapid innovation in the field 2.

   

Benchmarks

DBRX has shown impressive performance in various benchmarks. Jaeden explains that in language understanding, DBRX scored 73.7, outperforming models like LLaMA and Mixtral 3. In programming benchmarks, DBRX achieved a 70% score, significantly higher than its competitors 3.

They actually beat GPT-3.5. This is not a surprise. GPT-3.5 is getting completely smoked by GPT-4.

--- Jaeden Schafer

Jaeden acknowledges that while DBRX excels in open-source benchmarks, it still lags behind GPT-4, which remains the industry leader 1.

   

Development

The development of DBRX was remarkably swift and cost-effective. Jaeden praises Databricks for creating a competitive model in just two months with a budget of $10 million 4. He contrasts this with Apple's inability to develop a similar model despite their vast resources 5.

This is brand new. They cranked this thing out in two months for $10 million. So I'm gonna give them, like, a ton of, a ton of credit for this.

--- Jaeden Schafer

Jaeden also highlights Databricks' strategy of open-sourcing DBRX to attract talent and support their custom AI model business, which addresses security and compliance concerns for clients 5.