Plug and Play Language Models: A Simple Approach to Controlled Text\n Generation

Large transformer-based language models (LMs) trained on huge text corpora\nhave shown unparalleled generation capabilities. However, controlling\nattributes of the generated language (e.g. switching topic or sentiment) is\ndifficult without modifying the model architecture or fine-tuning on\nattribute-specific data and entailing the significant cost of retraining. We\npropose a simple alternative: the Plug and Play Language Model (PPLM) for\ncontrollable language generation, which combines a pretrained LM with one or\nmore simple attribute classifiers that guide text generation without any\nfurther training of the LM. In the canonical scenario we present, the attribute\nmodels are simple classifiers consisting of a user-specified bag of words or a\nsingle learned layer with 100,000 times fewer parameters than the LM. Sampling\nentails a forward and backward pass in which gradients from the attribute model\npush the LM's hidden activations and thus guide the generation. Model samples\ndemonstrate control over a range of topics and sentiment styles, and extensive\nautomated and human annotated evaluations show attribute alignment and fluency.\nPPLMs are flexible in that any combination of differentiable attribute models\nmay be used to steer text generation, which will allow for diverse and creative\napplications beyond the examples given in this paper.\n