Self correcting non-chronological autoregressive music generation

We describe a novel approach for generating music using a self-correcting, non-chronological, autoregressive model. We represent music as a sequence of edit events, each of which denotes either the addition or removal of a note—even a note previously generated by the model. During inference, we generate one edit event at a time using direct ancestral sampling. Our method allows the model to fix previous mistakes such as incorrectlysamplednotesandpreventtheaccumulation of errors which autoregressive models are prone to have. Another benefit is a finer, note-by-note control during human and AI collaborative composition. We show through human survey evaluation that our approach generates better results than orderless NADE and Gibbs sampling.