#039 - Lena Voita - NLP

Hallucination

In neural machine translation (NMT), hallucination occurs when models produce grammatically correct sentences that are disconnected from the source text. This happens because the model starts ignoring the input and relies on previously translated tokens, leading to errors. Elena Voita explains that exposure bias, where the distribution differs between training and inference, exacerbates this issue. During training, models use perfect tokens, but at inference, they rely on their own predictions, which can lead to unexpected situations 1.

Hallucination is when you just ignore the source because you want to produce something that's grammatically correct based on what you've already produced.

--- Unknown

Voita suggests that while some methods can mitigate exposure bias, hallucinations still occur, indicating a complex relationship between these phenomena 2.

   

Exposure Bias

Exposure bias in NMT models arises from the mismatch between training and testing inputs, known as teacher forcing. During training, models use the correct token, but during testing, they rely on their own predictions, leading to potential errors 3. This bias causes models to over-rely on target history, increasing the likelihood of hallucinations. Tim Scarfe discusses how beam search can help reduce hallucinations by providing simpler prefixes that make models more confident about the source 4.

Beam search translations are usually simpler than references, and indeed, when she compares them using the beam search prefixes, the model is more confident about the source, so it's hallucinating less as time goes on.

--- Tim Scarfe

This approach highlights the importance of balancing source and target contributions to improve translation accuracy.

   

Source-Target

Balancing source and target contributions in NMT is crucial for accurate translations. Elena Voita notes that models often fail to properly use source information, leading to reliance on target history and increased hallucinations 5. Beam search offers a solution by simplifying prefixes, which helps models maintain focus on the source text. This method reduces entropy and enhances model confidence, minimizing hallucinations 4.

Models trained with more data rely on the source more confidently, and the training process is non-monotonic.

--- Elena Voita

Understanding the dynamics of source-target contributions can lead to more effective NMT systems.