Energy-Based Diffusion Neural Sampler for Boltzmann Densities

Efficiency and sample quality are essential when drawing statistically independent samples from a Boltzmann-type distribution, which is desired in a wide range of scientific problems, such as generating equilibrium samples of many-body systems. Statistical methods like Monte Carlo and MCMC, or actual numerical methods like Molecular Dynamics, are promis- ing but computationally expensive, emerging a trend that leverages the data compression capacity of neural networks for efficient sampling. In this thesis, we propose ENERGY-BASED DENOISING ENERGY MATCHING (EnDEM) and BOOTSTRAP ENDEM (BEnDEM). The former one, EnDEM, is inspired by the current state-of-the-art Boltzmann neural sampler, DEM, by targeting a less noisy stochastic energy estimator which allows many potentials to further improve performance. While the latter one, BEnDEM, is built on top of EnDEM which improves its learning target by bootstrapping from the learned energy. Both EnDEM and BEnDEM are trained in a bi-level iterated scheme as iDEM, which includes a simulation-free inner loop training an energy-based diffusion sampler and an outer-loop that simulates the learned diffusion sampler to generate more informative samples to further improve the sampler, resulting in scalability to high dimensions. We evaluate EnDEM and BEnDEM on a suit of tasks ranging from synthetic energy functions to invariant n-body particle systems, demonstrating their stronger capacity compared with DEM. We also provide multiple possible ways for further improvement built on top of our models, demonstrating their potential to solve higher dimensional and more complex tasks in the future.