No Priors Ep. 11 | With Matei Zaharia, CTO of Databricks

Dolly's Creativity

Matei Zaharia, CTO of Databricks, highlights the creative capabilities of Dolly, an AI model that performs impressively with fewer parameters than its larger counterparts. He explains that Dolly excels in generating coherent text, such as stories or scientific abstracts, despite its smaller size 1. This challenges the traditional belief that creativity in AI requires vast amounts of data and parameters. The open-source nature of Dolly, inspired by models like Alpaca and Lama, underscores the potential of accessible AI development. Zaharia notes, "It's surprisingly good at just free form, like kind of fluent text generation" 1.

   

Instruction Following

The ability of smaller models like Dolly to follow instructions effectively is reshaping the AI landscape. Zaharia discusses how Dolly, with only 6 billion parameters, can perform tasks previously thought to require larger models 2. This capability, known as instruction following, allows the model to understand and execute tasks with minimal prompting. He states, "It's been pretty surprising to a lot of researchers the size of model that still gets you this kind of instruction following ability" 2. This advancement opens new possibilities for AI applications, making them more accessible and efficient.

   

Open Source Impact

Open source plays a crucial role in the development of AI models like Dolly, as Zaharia emphasizes. By leveraging open datasets, Dolly was developed to be both efficient and accessible, challenging the dominance of large, proprietary models 1. Zaharia reflects on the origins of Databricks, which began as a project to democratize data and machine learning, leading to innovations like Apache Spark 3. He remarks, "We were really excited to look at making it easier to do computation on large amounts of data" 3. This approach not only fosters innovation but also encourages collaboration across the AI community.