Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective
An MCMC perspective on constrained sampling for language models.
Emmanuel Anaya Gonzalez
•
•
1 min read
Flexible and Efficient Grammar-Constrained Decoding
Flexible and efficient grammar-constrained decoding.
Grammar-Aligned Decoding
Some existing works constrain possible next tokens to ensure grammaticality of generated output. But validity alone is not enough: a decoding algorithm should also preserve the …