SciRE-Solver: Accelerating Diffusion Models Sampling by Score-integrand Solver with Recursive Difference

TL;DR — We introduce the recursive difference method to calculate the derivative of the score function in the realm of DMs, and propose SciRE-Solver for accelerating sampling of DM.

Abstract

One downside of Diffusion models (DMs) is their slow iterative process. Recent algorithms for fast sampling are designed from the differential equations. However, in the fast algorithms, estimating the derivative of the score function evaluations becomes intractable due to the complexity of large-scale, well-trained neural networks. In this work, we introduce the recursive difference method to calculate the derivative of the score function networks. Building upon, we propose \emph{SciRE-Solver} with the convergence order guarantee for accelerating DMs sampling. Our proposed sampling algorithms attain SOTA FIDs in comparison to existing training-free sampling algorithms, under various number of score function evaluations (NFE). Such as, we achieve $3.48$ FID with $12$ NFE, and $2.42$ FID with $20$ NFE for continuous-time model on CIFAR-10; $1.79$ FID with $20$ NFE and $1.76$ FID with $100$ NFE for the pretrained model of EDM. Experiments demonstrate also that demonstrate that SciRE-Solver with multi-step methods can achieve high-quality samples on popular text-to-image generation tasks with only 6$\sim$20 NFEs.