Learning Generative Maps & the Rewards Guiding Them: Normalizing Flow Maps & Surrogate Reward Mixing

Learning a generative model gives us a sampler for a data distribution. Adapting that sampler to a downstream task requires rewarding preferential outcomes typically in the form of scalar rewards or rankings that the model must internalize. In this talk, I will discuss learning generative maps as well as new methodology to exploit (multiple) rewards at inference used to guide them. I will first introduce Normalizing Flow Maps (NFM), a new generative modeling class which combine few-step generation with exact likelihood evaluation by learning invertible transport maps along a generative probability path. NFMs are centered around two matching principles: Pointwise and distributional matching provide that provide complementary training objectives, while access to sample densities enables direct likelihood-based post-training. In the