Colin discusses the significant leap in model size, reaching 11 billion parameters, a two orders of magnitude increase. He highlights the challenges of cost associated with running inference on such large models, contrasting the ease of fine-tuning smaller models for free.
