728: Use Contrastive Search to get Human-Quality LLM Outputs — with Jon Krohn (@JonKrohnLearns)

Sampling Methods

Sampling methods in LLMs, such as top k and nuclear sampling, are pivotal for generating coherent and human-like text outputs. Jon Krohn explains that these methods rely on probability distributions to select words, resulting in varied outputs each time a model is run, unlike deterministic methods like greedy or beam search 1. This randomness is why tools like ChatGPT provide unique responses to the same query.

In this paradigm, the highest probability word will be selected most frequently, but not always more specifically, using the technical terminology of probability theory, we sample words in this sampling paradigm according to the probability distribution that we extract from the large language model that we're using.

--- Jon Krohn

Despite their effectiveness, Krohn notes that contrastive search offers superior results, making it a preferred choice for production environments.

   

Advanced Techniques

Top k and nuclear sampling are specific techniques within the broader sampling paradigm that enhance the fluency of LLM outputs. Krohn highlights that these methods support the generation of human-like text by leveraging the probability distribution of words 1. However, he emphasizes that contrastive search, introduced at the NeuRIps conference, surpasses these methods in producing the most human-like outputs.

Two specific and popular decoding approaches that leverage sampling are top k sampling and nuclear sampling.

--- Jon Krohn

Krohn encourages the use of contrastive search in production, as it consistently yields superior results.