Language Model Training

Stella discusses the innovative approach of training language models on computer code, emphasizing the importance of incorporating code into natural language training. She highlights the challenges faced during training and the impact of A 100 GPUs on model stability. Lukas shares a humorous anecdote about running out of disk space during the training process.