Language Model Data Curation

Sachin discusses the challenges of curating data for language models, highlighting the implicit ideologies introduced by quality filters. By analyzing high school newspapers, he uncovers unintended consequences in data inclusion and exclusion based on these filters.