Data Mixing Effects

Brandon discusses how mixing code and language data in training sets impacts model capabilities, revealing a semantic bleed effect. By ablating plant articles, the model's understanding of animals and natural places shifts, showcasing the intricate relationships within data representations.