Eli discusses the challenges of online learning and immediate feedback in improving models, highlighting the nascent state of the field. Jerod explores the retraining process at Google and Apple, mentioning the use of transfer learning for changing architectures without starting from scratch.