Guidance is how a diffusion model is made to follow its prompt more closely. A conditional diffusion model is given a condition, such as a class label or a text prompt, and asked for a sample that depicts it. Sampled in the plain way, it often returns images that are plausible but only loosely match the condition. Guidance fixes this with one change to the sampler. At every denoising step the network is asked twice, once with the condition and once without, and the step moves further in the direction in which the condition changed the answer. One number, the guidance scale, sets how much further. After the number of sampling steps, it is the setting that most changes what a diffusion model produces.

The effect is large. Dhariwal and Nichol (2021) trained a diffusion model on ImageNet images of $256\times256$ pixels, with the class label as an extra input. Sampled in the usual way, it scored an FID of 10.94. FID is a distance between the statistics of generated and real images, so lower is better (it is defined below). They then added one guidance term to every sampling step, without retraining the diffusion network, and the FID fell to 4.59. Soon after, Ho and Salimans (2022) got the same effect with no extra network at all. Text-to-image systems have used their version ever since. The Stable Diffusion pipelines of diffusers, Hugging Face's standard library of diffusion code, run it at a scale of 7.5 by default.

A new idea was needed because the obvious one fails. A language model is made to commit to its most likely tokens by lowering its temperature, which sharpens the distribution of the next token (§6.4). The diffusion analogs are to scale up the network's prediction or to add less noise at each step. Dhariwal and Nichol tried both and got blurry, smooth images. What works is Bayes rule, applied to gradients of log-densities, and the whole section follows from it.

We derive classifier guidance from Bayes rule, then classifier-free guidance from classifier guidance, and implement both. On a toy whose exact answers we can compute, we measure what the scale buys and what it costs. We then prove that guided samplers do not draw from the distribution they are usually said to draw from, and show what they do instead. The last parts cover the refinements that production systems use, and fine-tuning on rewards, which builds a similar push into the weights. Interviews for image, video, and audio generation teams ask about every one of these.

Four facts from earlier in the chapter carry the whole section.

• The forward process. It turns clean data $\mathbf{x}_0 \in \R^d$ into a noisy sample $\mathbf{x}_t = \alpha_t\mathbf{x}_0 + \sigma_t\boldsymbol\epsilon$ (§33.1). Here $\boldsymbol\epsilon \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ is standard Gaussian noise and $\alpha_t^2 + \sigma_t^2 = 1$. Time runs from clean data at $t = 0$ to pure noise at $t = T$ in discrete steps, or at $t = 1$ when time is continuous.
• The score. The score of a density $p$ is the gradient $\nabla_{\mathbf{x}}\log p(\mathbf{x})$. It is an arrow at every point, pointing toward higher density, and it does not depend on the density's normalizing constant (§33.3).
• Noise prediction is the score in other units. A noise predictor is a network trained to guess $\boldsymbol\epsilon$ from $\mathbf{x}_t$. The best possible one is $\boldsymbol\epsilon^\star = -\sigma_t\nabla\log p_t$, minus $\sigma_t$ times the score of the noisy data (§33.3).
• Samplers. A sampler turns a noise predictor into samples by stepping from $t = T$ down to $0$. The ancestral sampler of §33.2, called DDPM (denoising diffusion probabilistic models), adds fresh noise at every step. The deterministic sampler of §33.4, called DDIM (denoising diffusion implicit models), adds none. Each DDIM step predicts the clean data, $\hat{\mathbf{x}}_0 = (\mathbf{x}_t - \sigma_t\hat{\boldsymbol\epsilon})/\alpha_t$, and re-noises it to the next, smaller time with the same predicted noise $\hat{\boldsymbol\epsilon}$.

Conditional diffusion is denoising with one more input

A condition $c$ is what the sample should depict, such as a class label or a text prompt. Training data now come in pairs $(\mathbf{x}_0, c)$, an image with its label or its caption. The forward process ignores $c$ and noises $\mathbf{x}_0$ exactly as before. So once $\mathbf{x}_0$ is known, the noisy $\mathbf{x}_t$ says nothing more about $c$. In symbols, $c \to \mathbf{x}_0 \to \mathbf{x}_t$ is a Markov chain.

The noisy data of one condition have the density

In plain words, take the clean data of condition $c$ and blur every point with Gaussian noise of the size that $t$ sets. We write $p_t(\mathbf{x})$ for the same integral taken over all the data, with $p(\mathbf{x}_0)$ in place of $p(\mathbf{x}_0\mid c)$. It is the unconditional noisy density. We write $\varnothing$ for the null condition, which means "no condition at all".

Nothing new needs proving. For each fixed $c$ the loss is the simple loss of §33.2, on data drawn from $p(\mathbf{x}_0\mid c)$. A squared error is minimized by the conditional mean. The link between the best noise predictor and the score (§33.3) holds for every distribution, so it holds for $p_t(\cdot\mid c)$ too. In plain words, a conditional diffusion model is an ordinary one with the condition as an extra input, and at its optimum it outputs minus $\sigma_t$ times the score of the noisy class.

How $c$ enters the network does not matter in this section. A class label usually becomes a learned vector that is added to the embedding of the time. Dhariwal and Nichol's network does this. So do the Diffusion Transformer, or DiT, a Transformer that denoises patches of a compressed image (Peebles and Xie 2023), and the small network we train below. A text prompt is encoded by a text model and read through attention, as text conditioning explains.

With exact scores, sampling this model gives exactly $p(\mathbf{x}\mid c)$, and there would be nothing to fix. Trained networks fall short. Their samples are plausible images that often do not commit to the label or the prompt. This is an empirical finding, and its cause is still debated. Bradley and Nakkiran (2024) call the reasons unclear. Karras et al. (2024) argue that a network of limited capacity spreads probability into regions where data are rare and it has learned little. Guidance is the fix that works in practice, and deriving it takes one identity.

Bayes rule turns a classifier into a score

The identity says that the score of one class is the unconditional score plus the gradient of a classifier's log-probability. Here it is at two points, by hand, before the general statement.

Proof. Bayes rule (§0.4) gives $p_t(\mathbf{x}\mid c) = p_t(c\mid\mathbf{x})\,p_t(\mathbf{x})/p(c)$. Take the logarithm and the gradient in $\mathbf{x}$. The term $-\log p(c)$ does not depend on $\mathbf{x}$, so its gradient is zero. For the integral, $p(c\mid\mathbf{x}_t) = \int p(c\mid\mathbf{x}_0, \mathbf{x}_t)\,p(\mathbf{x}_0\mid\mathbf{x}_t)\,\mathrm{d}\mathbf{x}_0$. The Markov chain $c \to \mathbf{x}_0 \to \mathbf{x}_t$ gives $p(c\mid\mathbf{x}_0, \mathbf{x}_t) = p(c\mid\mathbf{x}_0)$. $\square$

In plain words, to turn an unconditional score into a conditional one, add the input gradient of a classifier that looks at the noisy image. The prior $p(c)$ vanishes under the gradient, just as the normalizing constant of a density does in §33.3.

The noisy posterior $p_t(c\mid\mathbf{x})$ is a different function at every noise level. At $t = 0$ it is the ordinary classifier on clean data. As $t$ grows, $\mathbf{x}_t$ says less and less about $\mathbf{x}_0$. So $p_t(c\mid\mathbf{x})$ flattens toward the prior $p(c)$, and its gradient shrinks. To use the theorem we need a classifier for every noise level.

Classifier guidance adds a classifier's gradient to the score

Classifier guidance takes the identity and turns up the classifier term. It adds the classifier's gradient to the unconditional score, multiplied by a number that is usually larger than one.

The noise form follows from $\boldsymbol\epsilon = -\sigma_t\mathbf{s}$. Multiply the guided score by $-\sigma_t$. The first term becomes $\boldsymbol\epsilon_\theta(\mathbf{x}_t, t)$, and the second becomes $-\gamma\sigma_t\nabla\log p_\phi$.

Forgetting the factor $\sigma_t$ is a common bug. At high noise $\sigma_t \approx 1$, so the bug changes little there. Near the end of sampling $\sigma_t$ is small, say 0.05, and the classifier term should be shrunk 20 times. Without the factor, guidance is 20 times too strong exactly where the fine detail of the image is decided. An interviewer who asks you to write this update will usually ask next where the $\sigma_t$ comes from.

What a scale above one aims for

The guided score is the gradient of $\log\big[p_t(\mathbf{x})\,p_\phi(c\mid\mathbf{x})^\gamma\big]$. The hope behind guidance is that the sampler then ends at the matching distribution of clean data. Using $p(c\mid\mathbf{x}) = p(\mathbf{x}\mid c)\,p(c)/p(\mathbf{x})$, that distribution is

which we call the $\gamma$-powered distribution. In plain words, raising the classifier to a power $\gamma > 1$ sharpens it. So $p_\gamma$ puts its mass where the classifier is most confident that a sample belongs to $c$.

The push past the class is a toy picture of what Ho and Salimans saw in images. Their strongly guided ImageNet samples have saturated colors, as if each class were exaggerated away from the others.

There is a catch, and a later part of this section is about it. Guidance does not power the clean data distribution once. At every step it powers the noisy distributions, $p_t(\mathbf{x}\mid c)^\gamma p_t(\mathbf{x})^{1-\gamma}$. Noising and powering do not commute, so this sequence of fields is not the sequence of noised versions of $p_\gamma$. The next cell checks this for the two Gaussians at the noise level $\alpha_t^2 = 0.5$.

Powering the noised densities gives $\mathcal{N}(1.414, 0.5)$. Noising the powered target gives $\mathcal{N}(1.061, 0.625)$. They are different distributions. So the field that guidance follows at this noise level is not the score of the noised target. We will see that neither the deterministic nor the stochastic sampler with guidance draws from $p_\gamma$.

Why the classifier must see noise

Bayes rule asks for $p_t(c\mid\mathbf{x}_t)$, the posterior given a noisy input, which is a different function at every $t$. An off-the-shelf classifier trained on clean images answers a different question. It also sees inputs unlike anything it was trained on, so its gradient at high noise is meaningless. Dhariwal and Nichol therefore trained their classifiers on the same noising distribution as the diffusion model. Their denoising network is a U-Net, a convolutional network that shrinks the image to coarse features and then expands it back. The classifier is the shrinking half of a U-Net, with an attention-pooling layer at the $8\times8$ resolution. At $256\times256$ it has 54 million parameters, against the generator's 554 million.

A second worry led to the next idea. The input gradient of a classifier can raise its confidence without making the image more like the class, which is how adversarial examples work. Ho and Salimans asked whether classifier guidance improves the image metrics partly by fooling classifiers, since those metrics are themselves computed by a classifier. Classifier-free guidance settles the question, because it uses no classifier gradient at all.

What classifier guidance does on ImageNet

Every guidance paper reports the same few metrics, and each one says something different.

Three things stand out. First, guiding the unconditional model with scale 1 made FID worse, although a perfect classifier would then give the exact conditional score. The authors report that the classifier gave the requested class about 50 percent probability, yet the samples did not look like the class. They needed a scale well above 1. Second, the best results came from guiding a model that was already conditional (FID 4.59, against 12.00 for the best unconditional row). The conditional network's own knowledge of the class is worth keeping. Third, apart from that failed run at scale 1, more guidance raises precision and Inception Score and lowers recall. Guidance buys fidelity with diversity, and an interviewer will expect you to say which metric moves which way.

Classifier-free guidance needs no classifier

Classifier guidance needs a second network, trained on noisy data, whose gradients might be adversarial. Ho and Salimans removed it by reading Bayes rule for scores backward. Solve the theorem for the classifier's gradient,

The right side needs no classifier, only the conditional and unconditional scores. It is the gradient of the log of the ratio $p_t(\mathbf{x}\mid c)/p_t(\mathbf{x})$, which they call the implicit classifier. Substitute it into classifier guidance,

and multiply by $-\sigma_t$ to get noise units.

Here is the rule with numbers first. Say the network's prediction without the prompt is $(0.2, -0.1)$, and with the prompt it is $(0.5, 0.3)$. The prompt moved the answer by $(0.3, 0.4)$. Guidance at $\gamma = 3$ takes three times that move from the unconditional point, $(0.2, -0.1) + 3\,(0.3, 0.4) = (1.1, 1.1)$.

In plain words, ask the network twice, once with the prompt and once without, and step further along the direction in which the prompt changed its answer.

Papers write the same rule in three ways. Mixing them up silently changes the strength by one.

So the classifier-free guidance paper's best FID at $128\times128$, at $w = 0.3$, is at $\gamma = 1.3$, and its $w = 0$ is no guidance at all.

import numpy as np

def cfg_combine(eps_cond, eps_uncond, gamma):
    raise NotImplementedError

def cfg_from_w(w):
    raise NotImplementedError

With exact scores, classifier guidance and classifier-free guidance are the same rule, because the implicit classifier is then the true noisy posterior. The next exercise checks this on two Gaussian classes, where the noisy posterior has a closed form.

import numpy as np

def classifier_guidance_eps(eps, x_t, c, w_t, b_t, sigma_t, gamma):
    return eps   # placeholder, no guidance

One network learns both predictions

The unconditional model costs almost nothing extra. Give the network one more value of the condition, a learned null vector for a class label or the encoding of the empty prompt for text. During training, replace the true condition by it at random. This is called condition dropout.

Ho and Salimans trained ImageNet $64\times64$ models with $p_{\text{uncond}}$ equal to 0.1, 0.2, and 0.5. The values 0.1 and 0.2 did about equally well. The value 0.5 was worse at every point of the trade-off between FID and Inception Score. Only a small share of the network's capacity needs to go to the unconditional task, and most systems use 0.1 to 0.2.

The price is paid at sampling time. The number of function evaluations (NFE) of a sampler is the number of network calls it makes per sample (§33.4). Every guided step needs two.

The CFG rule is written for noise predictions, but many networks predict something else, such as the clean image or a velocity. The rule still applies to their outputs whenever the output is an affine function of the noise prediction, a constant times the noise prediction plus a term that does not depend on it.

Proof. The left side is $(1-\gamma)(a_t\mathbf{x}_t + b_t\hat{\boldsymbol\epsilon}_\varnothing) + \gamma(a_t\mathbf{x}_t + b_t\hat{\boldsymbol\epsilon}_c)$. The weights $1 - \gamma$ and $\gamma$ sum to one, so the $a_t\mathbf{x}_t$ terms combine to $a_t\mathbf{x}_t$. What remains is $b_t$ times the guided noise. $\square$

The weights summing to one is the whole point. Multiplying a single prediction by $\gamma$, the literal temperature move, uses weights that do not sum to one, and it does not commute with these conversions. The theorem is why a model that predicts velocities, such as a rectified flow transformer (a Transformer trained to move noise to data along straight paths), can apply CFG to its own output with the same formula (§33.6).

import numpy as np

def drop_condition(c, p_uncond, rng, null_id=NULL):
    raise NotImplementedError

def cfg_ddim_sample(eps_model, c, gamma, x_T, alpha, sigma, ts, null_id=NULL):
    raise NotImplementedError

What the scale does, measured with exact scores

With images it is hard to separate what guidance does from what an imperfect network does. So we build a toy whose exact scores we can compute.

• Two classes live in $\R^2$, and each is a mixture of two Gaussians with standard deviation 0.5.
• Class A has modes at $(-1.6, 0)$ and $(0.3, 0)$, with equal weights.
• Class B has modes at $(1.4, 0.9)$ and $(1.4, -0.9)$, with weights 0.7 and 0.3.
• The two classes are equally likely.

Class A's right-hand mode, the near mode, sits close to class B. Its left-hand mode, the far mode, is far from class B, and that will matter.

A noised mixture of Gaussians is again a mixture of Gaussians, with means $\alpha_t\boldsymbol\mu_k$ and variances $\alpha_t^2\cdot0.5^2 + \sigma_t^2$ (§33.1). Its score has a closed form (§33.3). So $\boldsymbol\epsilon^\star(\mathbf{x}, t, c)$ and $\boldsymbol\epsilon^\star(\mathbf{x}, t, \varnothing)$ are exact, and any gap between the samples and the class comes from guidance and the sampler alone. We use DDPM's linear schedule with $T = 1000$, and deterministic DDIM with 100 steps.

The sampler below is DDIM with the CFG prediction. Its guide argument decides at which steps guidance is applied, which we use later for guidance intervals. Its neg argument replaces the unconditional pass by another condition, which we use later for negative prompts. Three numbers summarize a set of class-A samples.

• The precision here is the fraction of samples that the clean Bayes classifier assigns to class A.
• The spread is the trace of the sample covariance, the total variance over both coordinates.
• The far-mode share is the fraction of samples nearer to the far mode $(-1.6, 0)$ than to the near one. It is 0.5 for the true class.

Read the table row by row. At $\gamma = 0$ we sample the unconditional mixture, and only half the samples belong to class A. At $\gamma = 1$ the sampler reproduces class A, up to the small error of 100 DDIM steps (precision 0.935 against the exact 0.925, spread 1.37 against 1.40). The 7.5 percent of class-A samples that the classifier calls class B are not mistakes. They are genuine class-A points in the overlap, and an exact model draws them.

Above $\gamma = 1$ the two columns part ways. At $\gamma = 3$ CFG has precision 1.000, but its spread has fallen to 0.51, less than half of the target's 1.16. And 94.5 percent of its samples are in the far mode, where the powered target puts only 59 percent. At $\gamma = 7.5$ the near mode is gone and every sample is in the far mode, while the powered target still keeps 35 percent of its mass near $(0.3, 0)$.

The plot shows the samples on top of the powered target's contours.

At $\gamma = 7.5$ the samples have not just left the near mode. They have also slid left of the far mode's center, past it and away from class B, to a mean horizontal position near $-2.5$ against the center's $-1.6$. That is the overshoot of the hand example, now in two dimensions.

The widget runs the same toy in your browser. The dots are 500 samples of one class from DDIM with guidance and exact scores, and the gray shading is the powered target. Drag $\gamma$ and watch the two numbers on the right, the class precision and the share of samples in one mode, with the target's share beneath it.

Switch to class B, whose two modes are mirror images across the horizontal axis, so class A is equally close to both. There the powered target barely changes with $\gamma$, since its share in the heavier mode moves only from 0.69 toward 0.72. Yet CFG moves mass into the heavier mode as $\gamma$ grows, a change of mode weights that the target does not have. The slider for the guidance window and the option "A against B" are explained later in the section.

The same experiment with a trained network

With exact scores, $\gamma = 1$ is already perfect, and every $\gamma > 1$ is a distortion. With a trained network, $\gamma = 1$ is not perfect, and guidance makes up for some of the network's errors. The block below trains a small network with condition dropout ($p_{\text{uncond}} = 0.15$) on samples from the toy, then samples it with CFG. It also reports the off-manifold rate, the fraction of samples more than three standard deviations from every mode center.

In our run the trained model at $\gamma = 1$ reached a precision of only 0.88, below the exact 0.925. A small network trained for two thousand steps has not learned the class boundary exactly. Guidance at $\gamma = 3$ took precision to 1.000, which is what practitioners want from guidance. The costs match the exact toy's. The far-mode share rose well above 0.5, and at $\gamma = 7.5$ the near mode vanished.

A third cost appears only with a trained network. At large $\gamma$ the extrapolated prediction leaves the region where the network was trained, and some samples land far from every mode. In our run the off-manifold rate went from 0.7 percent at $\gamma = 1$ to about a quarter at $\gamma = 7.5$, though the exact number depends on the seed. This is the toy version of the oversaturated, overcooked images that high guidance scales produce.

What guided samplers actually sample

The most common explanation of CFG says that it samples the powered distribution $p_\gamma(\mathbf{x}\mid c) \propto p(\mathbf{x}\mid c)^\gamma p(\mathbf{x})^{1-\gamma}$. The toy showed that it does not. The next theorem says why, and what the exact guidance term would have to be. It covers any reweighting of the data, so we state it for a general weight function $h$.

Proof. By definition $\tilde p_t(\mathbf{x}) = \int q(\mathbf{x}\mid\mathbf{x}_0)\,p(\mathbf{x}_0)\,h(\mathbf{x}_0)\,\mathrm{d}\mathbf{x}_0/Z$. Multiply and divide by $p_t(\mathbf{x})$. By Bayes rule, $q(\mathbf{x}\mid\mathbf{x}_0)\,p(\mathbf{x}_0)/p_t(\mathbf{x}) = p(\mathbf{x}_0\mid\mathbf{x}_t = \mathbf{x})$. What remains is $p_t(\mathbf{x})\int p(\mathbf{x}_0\mid\mathbf{x}_t = \mathbf{x})\,h(\mathbf{x}_0)\,\mathrm{d}\mathbf{x}_0/Z$. Take the log and the gradient, and $Z$ drops out. $\square$

In plain words, to sample data reweighted by $h$, add to the score the gradient of the log of the weight you expect the clean image to have, given the noisy one. Three cases follow, each in a line.

• Bayes rule for scores is the case $h(\mathbf{x}_0) = p(c\mid\mathbf{x}_0)$. Then $\tilde p = p(\cdot\mid c)$ and $\mathbb{E}[p(c\mid\mathbf{x}_0)\mid\mathbf{x}_t] = p_t(c\mid\mathbf{x}_t)$, the theorem we started from.
• The powered target is the case $h(\mathbf{x}_0) = p(c\mid\mathbf{x}_0)^\gamma$. Its exact guidance term is $\nabla\log\mathbb{E}[p(c\mid\mathbf{x}_0)^\gamma\mid\mathbf{x}_t]$, the log of the expected power. Classifier guidance and CFG use $\gamma\nabla\log\mathbb{E}[p(c\mid\mathbf{x}_0)\mid\mathbf{x}_t]$, the log of the power of the expectation. For $\gamma > 1$ the map $u \mapsto u^\gamma$ is strictly convex, so by Jensen's inequality the first expectation is at least the second power. They are equal only when $p(c\mid\mathbf{x}_0)$ is constant over the posterior of $\mathbf{x}_0$. At high noise that posterior is wide, and the ratio of the two varies with $\mathbf{x}_t$, so the gradients of their logs differ. The guided field becomes the score of the noised target only as $t \to 0$, where the posterior collapses onto a point.
• Reward tilting is the case $h = \exp(r/\beta_{\text{KL}})$, for a reward $r(\mathbf{x}_0)$. Maximizing expected reward minus $\beta_{\text{KL}}$ times the KL divergence to a reference model has the solution $p_{\text{ref}}\exp(r/\beta_{\text{KL}})/Z$ (§17.1, which writes $\pi_{\text{ref}}$ and a plain $\beta$). Its exact guidance term is $\nabla\log\mathbb{E}[\exp(r(\mathbf{x}_0)/\beta_{\text{KL}})\mid\mathbf{x}_t]$, a soft value function of the noisy state. Estimating it is hard, which is one reason the last part of this section fine-tunes the network instead.

A case solved in closed form

Bradley and Nakkiran (2024) found a joint distribution for which the guided samplers can be solved exactly. Let $c \sim \mathcal{N}(0, 1)$ and $x\mid c \sim \mathcal{N}(c, 1)$ in one dimension, and condition on $c = 0$. Then $p(x\mid c = 0) = \mathcal{N}(0, 1)$ and $p(x) = \mathcal{N}(0, 2)$. By the precision rule of the hand example, the powered target is $\mathcal{N}\big(0, 2/(\gamma + 1)\big)$.

That last sentence should surprise you. In §33.3 the reverse SDE (the continuous-time limit of DDPM) and the probability flow ODE (the limit of DDIM) had the same marginals. That proof needed the field to be the score of the marginals of one forward process. The guided field is not such a score, so the proof does not apply, and the two samplers drift apart. Bradley and Nakkiran solve the variance exploding process. The derivation below solves the variance preserving one and gets the same answers.

The cell runs both samplers with exact scores on all 1000 steps of DDPM's schedule and compares them with the closed forms.

The simulated variances match the closed forms to within a few percent. At $\gamma = 3$, DDIM gives 0.249 against 0.250, and DDPM gives 0.377 against 0.388. DDPM's small gap comes from the discrete steps of the schedule and from Monte Carlo error with 20,000 samples. Neither is close to the target's 0.500. At $\gamma = 5$ DDIM's variance is 0.062, less than a fifth of the target's 0.333.

Guidance as a prediction step plus a sharpening step

If CFG does not sample $p_\gamma$, what is it doing? Bradley and Nakkiran give a clean answer for the DDPM version. It needs one tool from §33.3, Langevin dynamics. That is the update $\mathbf{x} \leftarrow \mathbf{x} + \eta\nabla\log\rho(\mathbf{x}) + \sqrt{2\eta}\,\mathbf{z}$, with step size $\eta$ and fresh noise $\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$. In the limit of small steps it leaves the distribution $\rho$ unchanged, so repeating it many times draws samples from $\rho$. A sampler that alternates a denoising step (the predictor) with Langevin steps (the corrector) is called a predictor-corrector sampler (§33.3).

In plain words, each guided DDPM step denoises toward the right class, then takes a single Langevin step toward a sharpened version of the current noisy distribution. One Langevin step per noise level is far too few to reach the sharpened distribution. And the sharpening at one level is inherited by the next. So the end result is shaped by the whole path, not by any single target.

In the closed-form example the result lands between the two powered distributions involved. Its variance at $\gamma = 3$ is 0.3875. That is between the $\gamma$-powered target's 0.5 and the $(2\gamma - 1)$-powered one's $2/(2\gamma) = 0.333$. This reading also explains the guidance interval later in the section. Sharpening a noisy distribution that is still one blob picks modes, which does more damage than sharpening one that has already separated into its modes.

The widget draws the exact outputs of both samplers, in the limit of many small steps, for the class $\mathcal{N}(0, 1)$ and an unconditional $\mathcal{N}(0, v_u)$ with $v_u$ from 1 to 4. Beside them it draws the $\gamma$-powered and the $(2\gamma - 1)$-powered distributions. The vertical scale stays fixed while $\gamma$ moves, so a curve that narrows also grows taller.

With $v_u = 2$, the closed-form case, move $\gamma$ from 1 to 6. The DDIM curve narrows exponentially, while the DDPM curve stays between the two powered ones. Then bring $v_u$ toward 1 and watch every curve converge. Guidance can only sharpen what distinguishes the class from the unconditional distribution.

import numpy as np

def cfg_ddim_variance(gamma, v_u, v_c, a2):
    return v_c                      # placeholder, as if CFG returned the class

def powered_variance(gamma, v_u, v_c):
    return v_c

What the scale trades at real scale

On ImageNet the scale traces out a curve, not a point. Ho and Salimans swept their $w$ from 0 to 4. They found the best FID at small guidance and the best Inception Score at the largest. Past the FID optimum, more guidance raised the Inception Score and worsened the FID at every step of the sweep.

FID compares the whole distribution of samples with the data, so it punishes the diversity that guidance removes. Its optimum sits at a gentle $\gamma$ between 1.1 and 1.3. Inception Score rewards confident, class-typical images and keeps rising. Which end you want depends on the product.

Refinements that fix the costs of plain guidance

Production systems rarely use plain CFG. Each refinement below fixes one cost of the plain version, and a table at the end of this part compares them.

Negative prompts replace the unconditional pass

A negative prompt $c^-$ describes what you do not want, such as "blurry, extra fingers, watermark". It replaces the unconditional pass,

In scores this is $\mathbf{s}_c + (\gamma - 1)\nabla\log\big[p_t(\mathbf{x}\mid c)/p_t(\mathbf{x}\mid c^-)\big]$. In plain words, it pushes toward images that the prompt explains much better than the negative prompt does. It costs nothing extra, since the negative pass replaces the unconditional one. When no negative prompt is given, implementations encode the empty prompt, which is $\varnothing$ again.

Be honest about what this is. The ratio $p_t(\mathbf{x}\mid c)/p_t(\mathbf{x}\mid c^-)$ compares two particular conditions. It is not the posterior of any classifier, so the Bayes argument behind CFG does not apply. The rule follows the field of the product $p(\mathbf{x}\mid c)^\gamma p(\mathbf{x}\mid c^-)^{1-\gamma}$, which can sit far from the data. Negative prompts are a heuristic that works well in practice, at moderate scales.

The widget's option "A against B" shows how far it can go wrong. With class B as the negative condition for class A, the samples at $\gamma = 3$ leave the data entirely, to a mean horizontal position near $-7$, and the product distribution goes with them. For two Gaussians of equal width that product is a Gaussian of the same width centered at $\gamma\boldsymbol\mu_c - (\gamma - 1)\boldsymbol\mu_{c^-}$, which moves linearly with $\gamma$.

Plain CFG cannot run away like this. Its unconditional density contains the class with weight $p(c)$, so the ratio $p(\mathbf{x}\mid c)/p(\mathbf{x})$ never exceeds $1/p(c)$. The push toward the class levels off once $\mathbf{x}$ is typical of it. A real negative prompt usually describes something that overlaps the prompt (the same scene, but blurry), which keeps the push local.

Rescaling stops oversaturation

At large $\gamma$ the guided prediction grows in magnitude. The images come out overexposed and oversaturated, and the effect is worst in a model with zero terminal SNR, one whose last noise level keeps no trace of the data (§33.1). The Imagen text-to-image model (Saharia et al. 2022) met the same problem in pixel space and fixed it with dynamic thresholding. At each sampling step, let $s$ be a high percentile of the absolute pixel values of the predicted $\hat{\mathbf{x}}_0$. If $s > 1$, clip $\hat{\mathbf{x}}_0$ to $[-s, s]$ and divide it by $s$. This pulls a strongly guided prediction back inside the $[-1, 1]$ range of the training images. A latent has no such fixed range, so Lin et al. (2023) proposed a fix in the same spirit that also works for latent models.

In plain words, shrink the guided prediction back to the spread the conditional prediction had, but only most of the way. Lin et al. apply this to the output of a v-predicting model. The diffusers library applies the same function to whatever the network outputs, noise or v, under the name guidance_rescale, which defaults to 0.

import numpy as np

def cfg_rescale(x_cfg, x_cond, phi):
    raise NotImplementedError

Guidance only where it helps

Kynkäänniemi et al. (2024) asked at which noise levels guidance does its good and its harm. They found it harmful at high noise, largely unnecessary at low noise, and beneficial only in the middle. So they apply it only for noise levels in an interval $(\sigma_{\text{lo}}, \sigma_{\text{hi}}]$ and use $\gamma = 1$ outside it. The noise level here is $\sigma = \sigma_t/\alpha_t$, the scale used by the EDM family of models (Karras et al. 2022), and it is tied to the log-SNR $\lambda = \ln(\alpha_t^2/\sigma_t^2)$ by $\lambda = -2\ln\sigma$ (§33.1).

Our toy shows the mechanism. The cell applies $\gamma = 4$ in different windows of time.

Guiding only the noisy steps ($t > 200$, log-SNR below 0.66) does all the damage. It gives the same mode collapse as guiding everywhere (far-mode share 0.993 against 0.995). Guiding only the window $100 < t \le 200$, log-SNR from 0.66 to 2.16, gives precision 1.000 with both modes intact (far-mode share 0.535). It also keeps most of the spread (1.04 against 1.37). Guiding only the cleanest steps, $t \le 100$, works too in this toy (precision 0.999, far-mode share 0.513, spread 1.20), because its two classes still overlap there and the boundary is still being drawn. On images, Kynkäänniemi et al. found guidance at low noise largely unnecessary. The toy shows the other half of their finding, that the damage is done at high noise.

The reason is the order of events. At high noise the noisy distribution is still one blob, and the guided field decides which mode each sample heads for. Extrapolating it there steers samples to the class's most distinctive mode. By the middle of the chain the modes have separated, and guidance only sharpens the boundary between classes, which is the part we wanted. In the widget, set $\gamma = 4$ and lower the upper end of the window from 1000 to 200. The far-mode share falls back from 99 percent to about 50, while the precision stays at 100 percent.

Guiding with a weaker copy of the model

CFG entangles two effects, better images and less variety. Karras et al. (2024) separated them with autoguidance. Instead of the unconditional model, they guide with a worse version of the same conditional model, smaller and trained for less time. Write $\boldsymbol\epsilon_1$ for the main model and $\boldsymbol\epsilon_0$ for this guide,

Their argument is that both models make the same kind of error in the same places, spreading into low-probability regions they have not learned, and the weaker one errs more. So their difference points from worse toward better. Extrapolating it corrects the main model's error without pushing away from other classes, and they report that it keeps much more variety than CFG.

The numbers are strong. With EDM2-S on ImageNet-512, a guide one size smaller (XS), trained on one sixteenth as many images, reached FID 1.34, against 2.23 for CFG and 2.56 with no guidance. EDM2-XXL guided by the medium-sized EDM2-M reached 1.25. Autoguidance also works for unconditional models, which CFG cannot guide at all, and took unconditional EDM2-S from FID 11.67 to 3.86. The price is a second model to train and store, though a small one.

Distilling guidance into one network call

CFG doubles the cost of every step. Meng et al. (2023) remove the doubling in two stages. First, they train a student $\boldsymbol\epsilon_{\theta'}(\mathbf{x}_t, t, c, w)$ that reproduces the teacher's guided prediction for a range of strengths $w$, in the $(1+w)$ convention. It takes the strength as an extra input, embedded with Fourier features as the time is, and it halves the network calls. Second, they apply progressive distillation (§33.4), which repeatedly trains a student to do two of the previous model's steps in one, to cut the number of steps.

Their pixel-space students for ImageNet $64\times64$ and CIFAR-10 came close to the teacher's FID and Inception Score with as few as 4 to 16 steps. Latent models such as Stable Diffusion produced good images in 1 to 4 steps. Released models now often ship with guidance distilled in. FLUX.1 [dev], a 12-billion-parameter rectified flow transformer, is distributed as a guidance-distilled model. Its model card says it was trained using guidance distillation, and its example passes guidance_scale=3.5 as an input to a single network call per step.

Guidance carries over to flow models

Flow-matching models (§33.6) predict a velocity, the direction a sample moves along its path from noise to data. For a Gaussian path the velocity is affine in the noise prediction at fixed $\mathbf{x}_t$. By the affine-target theorem, CFG applied to velocities is the same CFG. So are negative prompts, intervals, and autoguidance, which are all mixes with weights that sum to one. CFG rescale is the exception. A standard deviation does not commute with the affine conversion, so rescaling the velocity and rescaling the noise prediction give different results. An implementation must say which output it rescales.

Fine-tuning on rewards puts the push into the weights

Guidance pushes the sampler toward a classifier at sampling time. Often we want a push toward something no classifier provides, such as human preference, aesthetics, or correct object counts. And we would like it built into the weights, so that sampling stays cheap. The theorem on tilted targets says what the ideal is. For a reward $r(\mathbf{x}_0, c)$, the KL-regularized optimum of §17.1 is the tilt $p_{\text{ref}}(\mathbf{x}_0\mid c)\exp(r/\beta_{\text{KL}})/Z$, where $\beta_{\text{KL}}$ is the weight on the KL penalty. Three methods learn it, each by borrowing a tool from the language-model chapters.

Denoising as a sequence of decisions

DDPO (Black et al. 2024) treats denoising as a sequence of decisions, the setting of reinforcement learning (§15.3).

• The state is $(c, t, \mathbf{x}_t)$, and the action is $\mathbf{x}_{t-1}$.
• The policy is the learned reverse step $p_\theta(\mathbf{x}_{t-1}\mid\mathbf{x}_t, c)$.
• The only reward, $r(\mathbf{x}_0, c)$, arrives at the end.

Each reverse step is a Gaussian with mean $\boldsymbol\mu_\theta(\mathbf{x}_t, t, c)$ and fixed variance $\nu_t^2$. So its log-likelihood is exact and differentiable, $\log p_\theta(\mathbf{x}_{t-1}\mid\mathbf{x}_t, c) = -\lVert\mathbf{x}_{t-1} - \boldsymbol\mu_\theta\rVert^2/(2\nu_t^2) + \text{const}$.

The policy gradient raises the log-probability of each action in proportion to the reward that followed,

The version that reuses each batch of samples for several updates adds importance ratios with the clipping of PPO (§15.6), which limits how far each update can move the policy. DDPO fine-tuned Stable Diffusion for four rewards.

• JPEG compressibility, the file size after compression, and its opposite, incompressibility.
• The LAION aesthetics predictor, a model of how much people like an image, trained on 176,000 human ratings.
• Prompt alignment. A vision-language model (LLaVA) describes the image, and the description is compared with the prompt.

It also reported reward hacking of the kind §16.5 warns about, where the model finds an unintended way to score high. Asked for "$n$ animals", the fine-tuned model learned to write text that looks like the number, which fooled the vision-language model, instead of drawing the right count. Trained for incompressibility long enough, it stopped drawing anything recognizable and produced high-frequency noise.

Preferences become a comparison of denoising errors

Diffusion-DPO (Wallace et al. 2024) adapts direct preference optimization, DPO (§17.2). DPO learns from pairs of a preferred and a rejected output. It raises the policy's log-likelihood ratio to a frozen reference model on the preferred output, relative to the rejected one. That needs the log-likelihood of a sample, which a diffusion model does not have in closed form. So Wallace et al. replace it with the variational bound of §33.2, which turns log-likelihood ratios into differences of denoising errors.

Take a preferred image $\mathbf{x}_0^w$ and a rejected one $\mathbf{x}_0^l$. Noise each with its own noise, $\boldsymbol\epsilon^w$ and $\boldsymbol\epsilon^l$, to the same time $t$. For an image $\mathbf{x}$ noised with $\boldsymbol\epsilon$, let

the policy's denoising error minus the reference's. The loss is

where $\beta_{\text{DPO}}$ collects the KL weight, the number of steps, and a weighting, as the derivation below shows. (The prompt $c$ is an input to both networks and is left out of the notation.)

In plain words, get better than the reference at denoising the preferred image, and worse at denoising the rejected one. Here $\Delta < 0$ means the policy denoises that image better than the reference does. So the loss falls as $\Delta(\mathbf{x}^w, \boldsymbol\epsilon^w)$ goes down and $\Delta(\mathbf{x}^l, \boldsymbol\epsilon^l)$ goes up. When the policy equals the reference, every $\Delta$ is zero and the loss is $\log 2 = 0.693$.

Wallace et al. fine-tuned SDXL-1.0, an open latent text-to-image model, on Pick-a-Pic v2, a dataset of users' choices between pairs of generated images. After removing ties it has 851,293 pairs. On the PartiPrompts prompt set, human raters preferred the result 69 percent of the time over the larger SDXL pipeline with its refiner model. They found $\beta_{\text{DPO}}$ between 2000 and 5000 to work for both Stable Diffusion 1.5 and SDXL.

import numpy as np

def diffusion_dpo_loss(eps_model, eps_ref, xw, xl, t, eps_w, eps_l, alpha, sigma, beta_dpo):
    raise NotImplementedError

Making flow sampling random enough for reinforcement learning

GRPO (§18.4) samples a group of outputs per prompt and scores each one. Each output's advantage is its reward minus the group's mean, divided by the group's standard deviation, so no learned value function is needed. Flow-GRPO (Liu et al. 2025) brings it to flow-matching models. The obstacle is that their samplers solve an ODE, which is deterministic. A deterministic step has no probability to put in a policy gradient.

The fix uses a fact from the SDE view of §33.3. A whole family of reverse-time processes has the same marginals, from the deterministic probability flow ODE to the reverse SDE, indexed by how much noise they inject. Flow-GRPO converts the flow's ODE into one of these SDEs. The score the SDE needs follows from the predicted velocity, by the conversions of §33.6. Every step then becomes a Gaussian policy, and the sampler keeps the original marginals. A group of images per prompt can be sampled and scored. The policy is updated with clipped ratios and a KL penalty to the reference model, which is closed-form between Gaussian steps.

To save compute, Flow-GRPO collects training trajectories with only 10 denoising steps, while sampling with the full schedule at inference. It raised the GenEval accuracy of Stable Diffusion 3.5 Medium, an open rectified flow model, from 63 to 95 percent. GenEval checks object counts, colors, and positions with an object detector. Text-rendering accuracy rose from 59 to 92 percent. Without the KL term, image quality or diversity collapsed, which the authors describe as reward hacking. With a tuned KL term the gains held without that loss.

Rewards, preferences, and guidance are three handles on the same tilt. Guidance tilts at sampling time and pays in network calls. Fine-tuning tilts once, in the weights, and pays in training compute and in the reward-hacking risk that every learned reward carries.

Read more

• Dhariwal & Nichol (2021). Diffusion Models Beat GANs on Image Synthesis. Classifier guidance, its derivation as a mean shift, the noise-aware classifier, and the precision and recall trade-off in Table 4.
• Ho & Salimans (2022). Classifier-Free Diffusion Guidance. The implicit classifier, condition dropout, the $(1+w)$ convention, and the FID and Inception Score sweeps.
• Bradley & Nakkiran (2024). Classifier-Free Guidance is a Predictor-Corrector. The counterexamples showing that CFG does not sample the powered distribution, and the predictor-corrector reading.
• Kynkäänniemi et al. (2024). Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models. Why guidance at high noise drops modes, and the interval that fixes it.
• Karras et al. (2024). Guiding a Diffusion Model with a Bad Version of Itself. Autoguidance, and a careful analysis of why CFG improves image quality.
• Black et al. (2024). Training Diffusion Models with Reinforcement Learning. Denoising as a sequence of decisions, DDPO, and a vivid case of reward hacking.
• Wallace et al. (2024). Diffusion Model Alignment Using Direct Preference Optimization. Diffusion-DPO, derived from the variational bound.
• Liu et al. (2025). Flow-GRPO: Training Flow Matching Models via Online RL. The ODE-to-SDE conversion and GRPO for flow models.

Watch next

▶

▶

▶