Published Jun 29, 2023

141 - Building an open source LM, with Iz Beltagy and Dirk Groeneveld

Explore the revolutionary OLMo project with Iz Beltagy and Dirk Groeneveld as they delve into their mission to democratize language model development through open access and transparency, while discussing the impact of open-source methodologies, collaborative research, and innovative strategies for training models.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

115 - AllenNLP, interviewing Matt Gardner
Answers 383 questions
44 - Truly Low Resource NLP, with Anders Søgaard
Answers 383 questions
116 - Grounded Language Understanding, with Yonatan Bisk
Answers 383 questions
Are LLMs safe?
Answers 383 questions
109 - What Does Your Model Know About Language, with Ellie Pavlick
Answers 383 questions
40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis
Answers 383 questions
45 - Build It, Break It workshop, with Allyson Ettinger and Sudha Rao
Answers 383 questions
81 - BlackboxNLP, with Afra Alishahi and Tal Linzen
Answers 383 questions
136 - Including Signed Languages in NLP, with Kayo Yin and Malihe Alikhani
Answers 383 questions
128 - Dynamic Benchmarking, with Douwe Kiela
Answers 383 questions
119 - Social NLP, with Diyi Yang
Answers 383 questions
123 - Robust NLP, with Robin Jia
Answers 383 questions
27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov
Answers 383 questions
76 - Increasing In-Class Similarity by Retrofitting Embeddings with Demographics, with Dirk Hovy
Answers 383 questions
130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn
Answers 383 questions

Dexa/NLP Highlights

141 - Building an open source LM, with Iz Beltagy and Dirk Groeneveld

Topics covered

Popular Clips

OpenAI's Unique Approach

Allmo Project Overview

Language Model Recipe

Team Composition Impact

Advancing NLP Research

Language Model Training

Collaborative Project Success

Collaborative Efforts

Tackling Toxic Content

Training Language Models

Language Model Applications

Open Source Language Models

Collaboration Recommendations

Episode Highlights

Project Goals and Vision

Open Source Importance

Modeling Decisions

Related Episodes

115 - AllenNLP, interviewing Matt Gardner

44 - Truly Low Resource NLP, with Anders Søgaard

116 - Grounded Language Understanding, with Yonatan Bisk

Are LLMs safe?

109 - What Does Your Model Know About Language, with Ellie Pavlick

40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis

45 - Build It, Break It workshop, with Allyson Ettinger and Sudha Rao

81 - BlackboxNLP, with Afra Alishahi and Tal Linzen

136 - Including Signed Languages in NLP, with Kayo Yin and Malihe Alikhani

128 - Dynamic Benchmarking, with Douwe Kiela

119 - Social NLP, with Diyi Yang

123 - Robust NLP, with Robin Jia

27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov

76 - Increasing In-Class Similarity by Retrofitting Embeddings with Demographics, with Dirk Hovy

130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn

141 - Building an open source LM, with Iz Beltagy and Dirk Groeneveld

Topics covered

Popular Clips

Episode Highlights

Project Goals and Vision

Open Source Importance

Modeling DecisionsThe discussion with Iz Beltagy and Dirk Groeneveld explores the intricate process of training open-source language models. They share insights on optimizing training frameworks, team dynamics, and strategies for handling toxic content.

Modeling Decisions

Related Episodes