DBRX: The new best open LLM and Databricks' ML strategy

Efficiency

Nathan Lambert discusses the efficiency trends observed with Databricks' DBRX model. He highlights the significant performance gains achieved through meticulous training and engineering efforts. Lambert notes, "Our MoEs are two times more efficient than modern non-MoEs. Our data is two times more token efficient than for MPT. Inference is up to two times faster versus Llama 270B, up to 150 tokens per second" 1. These advancements suggest that GPT-4 level models could become effectively free within a decade.

   

Evaluation

Lambert also delves into the evaluation methods applied to DBRX, both qualitative and quantitative. He shares his experience testing the model's knowledge and limits, confirming that DBRX instruct is a solid sub-GPT-4 model. Lambert's attempts to jailbreak the model revealed consistent refusals, indicating robust safety filters 2 3. He concludes that while DBRX has some limitations, it remains a top contender in the open LLM space.