Researchers have introduced a new decoding method called Contrastive Search that can generate human-level text using off-the-shelf language models. The technique, presented at NeurIPS 2022 and detailed in a follow-up paper, addresses the shortcomings of existing decoding methods like greedy search and nucleus sampling.
Greedy search and beam search often produce repetitive and unnatural text, a problem known as model degeneration. For example, when prompted with "DeepMind Company is," a GPT-2 model using greedy search fell into a repetition loop. On the other hand, stochastic methods like nucleus sampling can avoid repetition but often generate semantically incoherent text, such as the unrelated phrase "AI is not journalism" in the same prompt.
Contrastive Search, now available in Hugging Face's Transformers library, aims to balance these issues. It selects tokens by contrasting between the original model and a perturbed version, encouraging the model to favor outputs that are plausible yet distinct from degenerate patterns. The method has been shown to work across 16 languages and with standard pretrained models.
A Hugging Face demo allows users to compare Contrastive Search with other decoding methods like beam search, top-k sampling, and nucleus sampling. For hands-on experience, a Colab notebook is provided.
The development marks a significant step in natural language generation, potentially improving applications from chatbots to content creation.