Training Challenges

Sewon Min discusses the challenges of training a retrieval model and solving the problem of predicting arbitrary length phrases without a decoder. He explains how a batch approximation is used to address the retrieval problem and how the start and end tokens are modeled separately to allow for arbitrary length phrases.